From the source
Hugging Face published a technical guide on token pooling-aware training, a method for shrinking multi-vector embedding models so they retrieve well from a smaller storage budget.
- Multi-vector models such as ColBERT keep a contextual vector per token, and a corpus of millions of documents can hold hundreds of millions of vectors, so the post walks through HierarchicalTokenPooling, which merges similar token vectors using Ward hierarchical clustering.
- The guide reports that a normally trained model with pool_factor=4 keeps 94.6% of its unpooled NanoBEIR nDCG@10 with a 4x reduction in document vectors, and that pooling-aware training raises that retention to 97.8% at the same storage budget.
- It also shows how to calibrate an adaptive distance threshold against a sample of the corpus to hit a target average vector count, such as 16 vectors per document, and notes the threshold can be reused at indexing time.
- The post builds on pooling-aware training studied by Stefan Josef and LightOn, and offers a short, reproducible Sentence Transformers example.





