# Ai2 — TutorMoments: Do AI tutors know when to help and when to hold back?

- Company: Ai2 (allenai.org)
- Announced: 2026-08-07
- Category: research-paper
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://allenai.org/blog/tutormoments
- Record: https://forck.live/items/18540-tutormoments-do-ai-tutors-know-when-to-help-and-when-to-hold-back
- Subject: Olmo / Molmo / Asta

Ai2 introduced TutorMoments, a framework and dataset for evaluating whether large language models can appropriately decide when to scaffold or push for rigor in math tutoring. Based on 462 real tutoring transcripts with over 1,500 teacher-annotated key moments, the evaluation found that models tend to over-help and rarely push students to deeper thinking, even when the trade-off is spelled out in the prompt. Ai2 released the de-identified transcripts, replay pipeline code, and model replays for reproducibility.

## Evidence

Verbatim from https://allenai.org/blog/tutormoments:

> TutorMoments is a replay-based evaluation built off real one-on-one math tutoring sessions. Experienced math teachers go through transcripts collected from a U.S. tutoring program and flag the moments where a tutor had to choose between making a problem easier to get started on and pushing the student to do more of the reasoning themselves.

---

Record: https://forck.live/items/18540-tutormoments-do-ai-tutors-know-when-to-help-and-when-to-hold-back
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md