# Ai2 — olmo-eval: An evaluation workbench for the model development loop

- Company: Ai2 (allenai.org)
- Announced: 2026-06-12
- Category: developer-tool-release
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://allenai.org/blog/olmo-eval
- Record: https://forck.live/items/18552-olmo-eval-an-evaluation-workbench-for-the-model-development-loop
- Subject: Olmo / Molmo / Asta

Ai2 released olmo-eval, an evaluation workbench for LLM development that builds on the OLMES standard. It offers more flexibility in defining evaluations, supports agentic and multi-turn evaluation, and provides analysis tools to compare checkpoints and distinguish real improvements from noise.

## Evidence

Verbatim from https://allenai.org/blog/olmo-eval:

> But a model's final score is only part of the evaluation process—which is why we're releasing olmo-eval , a new workbench that builds on OLMES and extends it across the rest of LLM development.

---

Record: https://forck.live/items/18552-olmo-eval-an-evaluation-workbench-for-the-model-development-loop
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md