From the source
Baseten replicated recent claims that the gains from full reinforcement learning runs can be recovered by linearly extrapolating a few early optimization steps, and found the effect holds only in narrow ways.
- The company trained Qwen2.5-Math-1.5B with GRPO for 500 steps and fitted a single rank-1 direction to the first k steps, following the RELEX method, to extrapolate to a future checkpoint with no further training.
- According to Baseten, scaling even just the first gradient step 1000x with Qwen2.5-1.5B on MATH exceeds a 500-step RL'd model, at 66.7% against 63.8%.
- Baseten says the improvements largely come from fixing one failure mode: fixing where the base model doom loops, and of 1,155 MATH questions the base model got wrong and extrapolation fixed, 72.0% of base rollouts showed heavy repetition.
- That effect weakens when off-distribution, as first step scaling does not help function calling or Knights & Knaves.
- Baseten reports the gains also vanish when scaling up to Qwen3-8B-Base.
- Baseten concludes that exploiting linear structure in RL trajectories may, in certain contexts, be about removing easy failure modes rather than gaining fundamentally new capabilities.




