Model Neuroscience: Dissecting Behavioral Change With Targeted Contrastive Neuron Attribution

Nous Research is publishing a method called Contrastive Neuron Attribution (CNA) that identifies sparse neuron circuits responsible for specific model behaviors, aiming to steer LLMs reliably without degrading performance.

• The technique builds on prior work showing that behavioral circuits live in MLP neurons rather than residual stream activations, and uses as few as eight pairs of contrastive prompts to find them.
• In tests on Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct, Llama-3.2-3B-Instruct and Qwen2.5-72B-Instruct, CNA reduced refusal behavior on unsafe requests while the source reports that base models show little change when steered the same way, suggesting refusal mechanisms develop during fine-tuning.
• The work does not yet cover Mixture-of-Experts models, and amplifying a discovered circuit still degrades quality while ablation does not, an open question the team plans to investigate.