From the source
Harvey introduced BigLaw Bench, a public version of its internal dataset for evaluating large language models on complex legal tasks.
- The benchmark uses bespoke rubrics to score model responses on real-world billable tasks, measuring both answer quality and source verifiability.
- Harvey reported that its proprietary models significantly outperform publicly available LLMs, but all models show substantial room for improvement compared to full completion of tasks performed by lawyers.
