Text embeddings explained: how they work [+examples]

Perplexity's explainer walks through how text embeddings turn words, sentences and documents into vectors that machine learning systems can compare.

• It describes the pipeline an embedding model runs on new text: tokenization into tokens with vocabulary IDs, encoding and pooling into a vector, then similarity calculation using measures such as cosine similarity or dot product.
• The piece distinguishes word, sentence and document embeddings by how much text each single vector represents, and notes that traditional models like Word2Vec learn by predicting a word from its surrounding words while modern retrieval-focused models often use contrastive learning.
• It lists business uses including semantic search, text classification, clustering, retrieval-augmented generation, recommendation systems, named entity recognition and machine translation, citing Delivery Hero, MindCraft.ai, Doczy.ai, Ring, LinkedIn's JUDE system, Amazon Health Services and Transglot as examples.