From the source
Baseten studied whether supervised fine-tuning on teacher solutions improves later reinforcement learning, comparing Qwen3 models at 0.6B, 1.7B, 4B and 8B parameters under GRPO across sixteen reasoning tasks.
- Distillation refers to supervised training on teacher responses, which the company describes as often used to warm-start a model so that reinforcement learning has a non-zero starting reward.
- Students were fine-tuned on teacher responses from GPT-5.6-Sol, with the main experiment using 2M response tokens before RL and three RL repetitions per starting condition at each size.
- SFT improved accuracy before RL at every size, by 31.2% at 0.6B against 15.3% at 8B, but the benefit after RL fell as students grew.
- After 80 million response tokens of RL, SFT + RL led RL by 11.3% at 0.6B, 13.1% at 1.7B and 2.1% at 4B, while at 8B RL alone beat SFT + RL by 6.4%.
- Baseten concludes that higher accuracy before RL does not necessarily yield higher accuracy after it, and that distillation loses value as the student gets larger.




