From the source
From the source
Aurora is an open-source, RL-based framework for adaptive speculative decoding that learns from live inference traces and continuously updates the speculator without interrupting serving, achieving an additional 1.25x speedup over a well-trained static speculator.
From the source
Today, we're releasing Aurora, an open-source, RL-based framework that learns from live inference traces and updates the speculator asynchronously—turning speculative decoding from a static, one-time setup into a dynamic, self-improving flywheel.
together.ai
