Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers

Sentence Transformers v6.0 introduces a new model type, MultiVectorEncoder, for ColBERT-style late interaction retrieval, along with a complete training approach.

The blog post demonstrates how to finetune multi-vector models that outperform general-purpose retrievers on domain-specific data, and shows that a finetuned model (multi-vector-encoder/mLateOn-medical) trained in 14.5 hours on a single RTX 3090 outperforms all general-purpose retrieval models on a medical retrieval evaluation.