From the source
How they respond matters too: whether they’re helpful, fair, safe, respectful, and responsive to the people using them.
For model builders, the challenge is knowing whether those “prosocial” behaviors hold up in practice—and whether evaluations capture how a model behaves when people interact with it in unexpected ways.
Northeastern University MS student Soham Padia used Olmo 3 to test whether crowdsourcing an evaluation of prosocial behavior could expose weaknesses that a small research team might miss.
Padia had developed an evaluation that measures how strongly text steers a model toward more prosocial responses.
Steering Arena turned that evaluation into a sort of game—players submit short text prefixes designed to influence the model, see how strongly each one shifts Olmo 3 in that direction, and compete for the top spot on the leaderboard.
Olmo’s openness made the project possible—Padia could see how submitted text changed Olmo 3’s internal activity instead of inferring those effects only from the responses it generated.
That access became the foundation for both his evaluation and Steering Arena.
…
