# Hugging Face — Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval

- Company: Hugging Face (huggingface.co)
- Announced: 2024-03-22T00:00:00+00:00
- Subject: Platform
- Source: https://huggingface.co/blog/embedding-quantization
- Record: https://forck.live/items/1920-binary-and-scalar-embedding-quantization-for-significantly-faster-cheaper

Hugging Face introduces binary and scalar embedding quantization techniques to improve retrieval speed, reduce memory usage, and lower costs. The methods involve converting float32 embeddings to 1-bit (binary) or int8 (scalar) values, enabling 32x reduction in memory. A demo with 41 million Wikipedia texts demonstrates practical benefits. Experiments show up to ~96% performance retention with rescoring.

## Evidence

Verbatim from https://huggingface.co/blog/embedding-quantization:

> We introduce the concept of embedding quantization and showcase their impact on retrieval speed, memory usage, disk space, and cost.

---

Record: https://forck.live/items/1920-binary-and-scalar-embedding-quantization-for-significantly-faster-cheaper
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md