From the source
Apple researchers introduced CapQuiz, a reference-free benchmark for evaluating video caption quality in Visual Large Language Models, along with the CapF1 composite metric.
- The benchmark uses human-verified multiple-choice questions across 10 question types and 24 video domains, and experiments show it correlates better with human judgments than existing metrics.