# Hugging Face — Token Pooling-Aware Training for Multi-Vector Embedding Models

- Company: Hugging Face (huggingface.co)
- Announced: 2026-11-11
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://huggingface.co/blog/token-pooling
- Record: https://forck.live/items/34643-token-pooling-aware-training-for-multi-vector-embedding-models
- Subject: Platform
- Models affected: NanoBEIR, ColBERT, lightonai/LateOn

Hugging Face published a technical guide on token pooling-aware training, a method for shrinking multi-vector embedding models so they retrieve well from a smaller storage budget. Multi-vector models such as ColBERT keep a contextual vector per token, and a corpus of millions of documents can hold hundreds of millions of vectors, so the post walks through HierarchicalTokenPooling, which merges similar token vectors using Ward hierarchical clustering. The guide reports that a normally trained model with pool_factor=4 keeps 94.6% of its unpooled NanoBEIR nDCG@10 with a 4x reduction in document vectors, and that pooling-aware training raises that retention to 97.8% at the same storage budget. It also shows how to calibrate an adaptive distance threshold against a sample of the corpus to hit a target average vector count, such as 16 vectors per document, and notes the threshold can be reused at indexing time. The post builds on pooling-aware training studied by Stefan Josef and LightOn, and offers a short, reproducible Sentence Transformers example.

## Evidence

Verbatim from https://huggingface.co/blog/token-pooling:

> Multi-vector embedding models give each query token its own opportunity to find a match in a document. This makes them wonderfully expressive for search, but those document vectors add up quickly. A corpus with millions of documents can contain hundreds of millions of vectors!
> 
> Token pooling makes these representations smaller by merging similar token vectors. The more aggressively I compress them, however, the easier it becomes to lose something useful. Pooling-aware training teaches the encoder to preserve the information that matters when its vectors are merged, helping you retrieve better from the same storage budget.
> 
> In this post, I'll explain how pooling works, train a model for compressed retrieval with Sentence Transformers, and explore the resulting quality and storage tradeoff. With pool_factor=4 , my normally trained model retains 94.6% of its unpooled NanoBEIR nDCG@10, with a 4x reduction in document vectors. Token pooling-aware training raises that retention to 97.8% at the same storage budget! Let's see how.

---

Record: https://forck.live/items/34643-token-pooling-aware-training-for-multi-vector-embedding-models
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md
