# Baseten — The Unreasonable Efficiency of a Single Gradient Step

- Company: Baseten (baseten.co)
- Announced: not stated by the source (first seen 2026-11-06T18:26:27.089748+00:00)
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://www.baseten.co/blog/the-unreasonable-efficiency-of-a-single-gradient-step
- Record: https://forck.live/items/32765-the-unreasonable-efficiency-of-a-single-gradient-step
- Subject: Baseten
- Models affected: Qwen2.5-1.5B, Qwen2.5-Math-1.5B, Qwen3-4B-Base, Qwen3-8B-Base
- Context window: 4,096-token context

Baseten replicated recent claims that the gains from full reinforcement learning runs can be recovered by linearly extrapolating a few early optimization steps, and found the effect holds only in narrow ways. The company trained Qwen2.5-Math-1.5B with GRPO for 500 steps and fitted a single rank-1 direction to the first k steps, following the RELEX method, to extrapolate to a future checkpoint with no further training. According to Baseten, scaling even just the first gradient step 1000x with Qwen2.5-1.5B on MATH exceeds a 500-step RL'd model, at 66.7% against 63.8%. Baseten says the improvements largely come from fixing one failure mode: fixing where the base model doom loops, and of 1,155 MATH questions the base model got wrong and extrapolation fixed, 72.0% of base rollouts showed heavy repetition. That effect weakens when off-distribution, as first step scaling does not help function calling or Knights & Knaves. Baseten reports the gains also vanish when scaling up to Qwen3-8B-Base. Baseten concludes that exploiting linear structure in RL trajectories may, in certain contexts, be about removing easy failure modes rather than gaining fundamentally new capabilities.

## Evidence

Verbatim from https://www.baseten.co/blog/the-unreasonable-efficiency-of-a-single-gradient-step:

> RL has become a necessary component for training frontier language models, but is also incredibly expensive. If the data, problems, and environments LLMs learn over are highly structured, then surely the weight dynamics for RL training are structured in some exploitable way too. A number of recent papers seem to indicate a rather striking structure result on RL optimization: they are effectively straight. Extrapolating from an early stretch of RL training can recover much of the improvement from a substantially longer run [ 1 , 2 , 3 ].
> 
> While many of these papers differ in their specific construction of a straight or low-rank trajectory, they paint a unifying picture: the same gains from a full RL run can be achieved by linearly extrapolating a few early optimization steps. This seems almost too good to be true.
> 
> In this short release update we replicate this claim and find that unfortunately, in some crucial ways, it kind of is. We find:
> 
> Scaling even just the first gradient step 1000x with Qwen2.5-1.5B on MATH exceeds a 500-step RL’d model (66.7% vs 63.8%).
> 
> The observed improvements largely come from fixing one failure mode, namely fixing where the base model doom loops.

---

Record: https://forck.live/items/32765-the-unreasonable-efficiency-of-a-single-gradient-step
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md
