Fast Embeddings on GPUs

Fast and accurate search is vital to all of Perplexity, from Search and Computer to our API Platform.

• Behind the scenes, the heavy lifting is done by embedding and ranking models, which help our systems identify the most relevant results for a given query.
• We achieve state-of-the-art quality and latency by training and serving our own models, such as pplx-embed .
• This article presents an under-the-hood view of Perplexity's serving infrastructure for this special class of models.
• We discuss our techniques to efficiently address the inference needs of AI-native search, enabling quick prototyping and evaluation of models while powering our exabyte-scale search index .
• These techniques collectively expand the Pareto frontier of search quality and efficiency, enabling us to serve agents and users with the best possible results at the lowest cost and latency.
• Embeddings for Search In a typical search setup, indexed documents are mapped to a high-dimensional vector space using an embedding model and stored in a vector database.
• By embedding a query using the same model, similar documents can be located by finding the vectors closest to that of the query.
• …