# Harvey — A Framework for Domain-Specific Evaluations

- Company: Harvey (harvey.ai)
- Announced: 2025-10-27T13:00:00+00:00
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://www.harvey.ai/blog/a-framework-for-domain-specific-evaluations
- Record: https://forck.live/items/17660-a-framework-for-domain-specific-evaluations
- Subject: Harvey

Harvey is expanding its public evaluation work into four areas: Insights, Research, Approaches, and Context. The company argues that the most important missing piece in LLM evaluation is a shared understanding of what evaluation does and does not tell us, and that misaligned benchmarks risk pushing models toward unhelpful behaviors. Harvey plans to publish at least two novel benchmarks: Contract Intelligence and a BLB Challenge benchmark focused on tasks where leading models struggle.

## Evidence

Verbatim from https://www.harvey.ai/blog/a-framework-for-domain-specific-evaluations:

> Today, we think the most important piece missing from the evaluation discourse is not a research finding or benchmark, but rather a clear, shared understanding of what evaluation of LLMs does and does not tell us . Failing to build a communal understanding of evaluation risks evaluation succumbing to Goodhart’s law: "When a measure becomes a target, it ceases to be a good measure."

---

Record: https://forck.live/items/17660-a-framework-for-domain-specific-evaluations
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md