From the source
Challenges in model routing for agentic systems, showing that actual cost depends on caching, difficulty is often invisible at routing time, and latency is affected by infrastructure.
The authors propose treating routing as an optimization problem rather than classification, and present results on AppWorld Test Challenge showing a cost-accuracy frontier with configurations achieving 84% accuracy at $93 and 83s latency, a 21% cost reduction and 9% latency reduction compared to running Opus alone with only a 4% accuracy drop.
