Lead story
Models & availability
Latest
← Feed
Copy✕ Close copy menu
Separating signal from noise in coding evaluations
OpenAI published an analysis highlighting problems with the SWE-Bench Pro coding benchmark, questioning its reliability and accuracy for evaluating AI models.
From the source
A new analysis from OpenAI reveals issues in SWE-Bench Pro, a popular coding benchmark, raising concerns about reliability and accuracy in evaluating AI models.
openai.com
Visit source ↗openai.com
- Announced
- 08 Jul 2026 13:00 UTC
- Detected
- 03 Aug 2026 21:11 UTC (+26d after)