Rethinking Neural Thickets Without Weight Perturbations

Each line is one perturbed model. Top: the shift of the logits builds up over the 32 decoder blocks. Bottom: it is added at the output bias. Both give a similar spread of logit shifts and, after selection and voting, a similar test accuracy; the dashed mark on each gauge is the unperturbed model. Schematic.

Perturbing logits is enough

A random bias on the output logits matches full-weight perturbation: MATH-500 goes from 63.3 to 81.3 on OLMo-3-7B-Instruct, against 74.7.

Weight noise acts through the logits

To first order, a weight perturbation is a shift of the logits. At the noise scale used, this reproduces 89.7% of argmax changes.

This carries over to vision

Under domain shift, perturbing the logits of a DINOv3 probe adds 6.2 points of worst-region accuracy on FMoW; perturbing the weights adds 1.0.

1 · Background Neural Thickets and perturb, select, vote

Neural Thickets improves a pretrained language model without training. Gan and Isola add random noise to its weights, keep the noisy copies that score best on a small labeled set, and let them vote.

We refer to this procedure as perturb, select, vote (PSV).