# Ai2 — DiScoFormer: One transformer for density and score, across distributions

- Company: Ai2 (allenai.org)
- Announced: 2026-06-29
- Category: research-paper
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://allenai.org/blog/discoformer
- Record: https://forck.live/items/18548-discoformer-one-transformer-for-density-and-score-across-distributions
- Subject: Olmo / Molmo / Asta

Ai2 introduced DiScoFormer, a transformer that estimates both the density and score of a distribution from a set of data points in a single forward pass without retraining. It outperforms kernel density estimation in high dimensions, cutting score error by about 6.5x and density error by more than 37x in 100 dimensions, and generalizes to out-of-distribution inputs via a label-free consistency loss.

## Evidence

Verbatim from https://allenai.org/blog/discoformer:

> Across the board, DiScoFormer beats KDE at both density and score estimation, and the gap widens exactly where KDE struggles. In 100 dimensions, it isn't close—against the best hand-tuned KDE, it cuts score error by about 6.5x and density error by more than 37x, and it keeps improving as you add samples, while KDE runs out of memory.

---

Record: https://forck.live/items/18548-discoformer-one-transformer-for-density-and-score-across-distributions
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md