# Kyutai — Rethinking Neural Thickets Without Weight Perturbations

- Company: Kyutai (kyutai.org)
- Announced: 2026-10-28
- Category: research-paper
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://kyutai.org/logit-psv
- Record: https://forck.live/items/29789-rethinking-neural-thickets-without-weight-perturbations
- Subject: Moshi / Pocket TTS
- Models affected: OLMo-3-7B-Instruct, Qwen2.5, DINOv3

Kyutai researchers report that perturbing only the output logits, rather than every weight, matches or beats full-weight Neural Thickets on most reasoning tasks. Neural Thickets improves a pretrained model without training by adding Gaussian noise to its weights, keeping the copies that score best on a small labeled set, and letting them vote. The team keeps that perturb, select, vote procedure but moves the noise to a single Gaussian vector over the vocabulary added to the logits at every decoding step, which amounts to perturbing the output-layer bias. On MATH-500, logit PSV takes OLMo-3-7B-Instruct from 63.3 to 81.3, against 74.7 for full-weight PSV. Across four models, logit PSV is more accurate than full-weight PSV on 13 of 15 reasoning cells and within 0.6 points on the other two, though it is usually slightly lower on MBPP and ROCStories. The authors say most of these differences are within test-set noise, and that a weight perturbation is to first order a logit shift, reproducing 89.7% of argmax changes at the noise scale used.

## Evidence

Verbatim from https://kyutai.org/logit-psv:

> Each line is one perturbed model. Top: the shift of the logits builds up over the 32 decoder blocks. Bottom: it is added at the output bias. Both give a similar spread of logit shifts and, after selection and voting, a similar test accuracy; the dashed mark on each gauge is the unperturbed model. Schematic.
> 
> Perturbing logits is enough
> 
> A random bias on the output logits matches full-weight perturbation: MATH-500 goes from 63.3 to 81.3 on OLMo-3-7B-Instruct, against 74.7.
> 
> Weight noise acts through the logits
> 
> To first order, a weight perturbation is a shift of the logits. At the noise scale used, this reproduces 89.7% of argmax changes.
> 
> This carries over to vision
> 
> Under domain shift, perturbing the logits of a DINOv3 probe adds 6.2 points of worst-region accuracy on FMoW; perturbing the weights adds 1.0.
> 
> 1 · Background Neural Thickets and perturb, select, vote
> 
> Neural Thickets improves a pretrained language model without training. Gan and Isola add random noise to its weights, keep the noisy copies that score best on a small labeled set, and let them vote.
> 
> We refer to this procedure as perturb, select, vote (PSV).

---

Record: https://forck.live/items/29789-rethinking-neural-thickets-without-weight-perturbations
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md