Fixing Open LLM Leaderboard with Math-Verify

Hugging Face re-evaluated all 3,751 models on the Open LLM Leaderboard using the Math-Verify tool to fix math evaluation issues, resulting in an average 4.66-point score increase and significant reshuffling of rankings, especially for Qwen and DeepSeek models.

From the source
we've used Math-Verify to thoroughly re-evaluate all 3,751 models ever submitted to the Open LLM Leaderboard, for even fairer and more robust model comparisons!
huggingface.co

Visit source ↗huggingface.co

Announced: 14 Feb 2025 00:00 UTC
Detected: 03 Aug 2026 21:12 UTC (+535d after)

Other stories

GitHub · 31 AugGitHub Copilot in VS Code, August 2026 releasesCapability change

xAI · 29 AugGrok Bot now works with XCapability change

GitHub · 28 AugGitHub Copilot weekly releases — August 24Capability change

Amazon · 28 AugSpreading the load: How Salesforce met Multi-AZ HA with SageMaker Inference ComponentsCapability change

Hugging Face · 28 AugThe Open ASR Leaderboard Adds Its First Global South LanguageCapability change

Amazon · 27 AugBuild agentic creative workflows with Amazon Quick and falCapability change