From the source
Large Reasoning Models (LRMs) employ a novel paradigm known as test-time scaling, leveraging reinforcement learning to teach the models to generate extended chains of thought (CoT) during reasoning tasks.
- This enhances their problem-solving capabilities beyond what their base models could achieve independently.
- While cost and efficiency trade-off curves ("the Pareto frontier") typically focus on model intelligence versus cost per million completion tokens, token efficiency — the number of tokens used for reasoning relative to the solution — is a critical factor that is recently receiving more attention.
- Anecdotal evidence suggests open weight models produce significantly more tokens for similar tasks than closed weight models.
- This report systematically investigates these observations.
- We confirm this trend to be generally true, but observe significant differences depending on problem domain.
- TL;DR: Closed models (
OpenAI, Grok-4) optimize for fewer tokens to cut costs, while open models (
DeepSeek, Qwen) use more tokens, possibly for better reasoning.
- …