From the source
Apple researchers presented a paper at ICML 2026 that trains small neural networks offline to route queries toward their best match faster in maximum inner product search.
- MIPS sits at the core of recommendation systems, search and retrieval-augmented generation, where a query vector is compared against millions of candidate key vectors to find the best match.
- As key counts reach millions and vector dimensions reach thousands, exhaustive comparison becomes dramatically more expensive, and existing approximate methods trade accuracy for speed using clustering and quantization.
- Those baselines focus on the geometry of the key set and loosely assume queries resemble keys, which the researchers say is problematic in RAG, where queries are short user questions and keys are long curated text.
- The work builds on the observation that the function mapping a query to its largest inner product is the support function of the key set, a convex object whose gradient returns the best-matching key.
- Two networks are trained on examples drawn from the relevant query distribution: SupportNet learns the support function with an input convex neural network, and KeyNet outputs the best-matching key vector directly, kept geometrically grounded by a consistency loss from Euler's theorem.
- The researchers report that training separate specialized networks for the two tasks works best in practice.
