When a model reads a drug's class from its name—not its knowledge

A study of Ai2's Olmo 3, accepted to EMNLP, found that 51–59% of tested drugs elicited little sign of drug-specific knowledge, while another 12–18% appeared affix-driven.

The researchers traced this behavior to the model's open training data, showing that rarer drugs in training led to greater reliance on name affixes rather than learned facts.

The study used Olmo 3 7B Instruct and the infini-gram engine for exact phrase lookup across the training corpus.