# Ai2 — Which tokens does a hybrid model predict better?

- Company: Ai2 (allenai.org)
- Announced: 2026-06-25
- Category: research-paper
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://allenai.org/blog/hybrid-token-prediction
- Record: https://forck.live/items/18549-which-tokens-does-a-hybrid-model-predict-better
- Subject: Olmo / Molmo / Asta
- Models affected: Olmo Hybrid, Olmo 3

Ai2 published research comparing its 7B transformer Olmo 3 and hybrid model Olmo Hybrid at the token level. The hybrid predicts content words such as nouns, verbs, and adjectives better than the transformer, but its advantage nearly disappears on tokens that repeat verbatim from earlier in the input, where the transformer's attention mechanism excels.

## Evidence

Verbatim from https://allenai.org/blog/hybrid-token-prediction:

> Our results show that the hybrid's advantage is real across many tokens—but not all. Olmo Hybrid is strongest on tokens that carry meaning, such as nouns, verbs, and adjectives, and on tokens that can only be predicted by following what's going on—like which person a pronoun refers to. But the hybrid's advantage almost disappears on tokens that simply repeat something already in the input – a word or phrase reproduced verbatim from earlier – where the answer is sitting right there to be looked up.

---

Record: https://forck.live/items/18549-which-tokens-does-a-hybrid-model-predict-better
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md