# Nous Research — Measuring Thinking Efficiency in Reasoning Models: The Missing Benchmark

- Company: Nous Research (nousresearch.com)
- Announced: 2025-08-14T18:07:45+00:00
- Category: not stated
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://nousresearch.com/measuring-thinking-efficiency-in-reasoning-models-the-missing-benchmark
- Record: https://forck.live/items/24030-measuring-thinking-efficiency-in-reasoning-models-the-missing-benchmark
- Subject: Hermes models

Large Reasoning Models (LRMs) employ a novel paradigm known as test-time scaling, leveraging reinforcement learning to teach the models to generate extended chains of thought (CoT) during reasoning tasks. This enhances their problem-solving capabilities beyond what their base models could achieve independently. While cost and efficiency trade-off curves ("the Pareto frontier") typically focus on model intelligence versus cost per million completion tokens, token efficiency — the number of tokens used for reasoning relative to the solution — is a critical factor that is recently receiving more attention. Anecdotal evidence suggests open weight models produce significantly more tokens for similar tasks than closed weight models. This report systematically investigates these observations. We confirm this trend to be generally true, but observe significant differences depending on problem domain. TL;DR: Closed models (OpenAI, Grok-4) optimize for fewer tokens to cut costs, while open models (DeepSeek, Qwen) use more tokens, possibly for better reasoning. …

---

Record: https://forck.live/items/24030-measuring-thinking-efficiency-in-reasoning-models-the-missing-benchmark
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md