From the source
Trust is not.
The system only makes progress as fast as we can check those drafts.
A number that looks miraculously fast is often a measurement bug: leftover work from a previous run, a comparison where the two sides were not doing the same thing, or a kernel that only looks good under a hidden assumption.
Context is a tradeoff, not a pile to maximize.
More text gives the model more to work with, but it costs more, and extra notes make it easier for the next attempt to drift.
Too little and the loop cannot move.
An agent that can explore freely writes better kernels.
A strict outer system has to define the feedback and decide what is allowed to ship.
A good design needs both.
Generated individual Qwen 3.5 122B kernels were 1.8–5.2× faster than the best implementations available in vLLM.
Traditionally, production inference systems rely on generic kernels to handle diverse models and workloads.
…
