Ai2· 12 Junolmo-eval: An evaluation workbench for the model development loop
Ai2 releases olmo-eval, an evaluation workbench for model development
Developer tool
Ai2 released olmo-eval, an evaluation workbench for LLM development that builds on the OLMES standard.
It offers more flexibility in defining evaluations, supports agentic and multi-turn evaluation, and provides analysis tools to compare checkpoints and distinguish real improvements from noise.
