Moonshot AI releases PerceptionBench, a benchmark for evaluating atomic visual perception in multimodal large language models, with 3,000 verified questions across 10 atomic perceptual categories, and open-sources the dataset and evaluation code.
From the source
We are releasing PerceptionBench , a benchmark that isolates visual perception and evaluates it as a set of atomic capabilities— discovered from how today's models fail, not defined in advance.