From the source
Harvey describes how it uses ELO scores, derived from lawyer preference evaluations, to measure and guide improvements to its AI systems.
The post explains that an ELO gap of 150–200 points reflects a clear preference (about 70% preference rate), while a 400-point gap marks a step-change.
Examples include building an Australia knowledge source and rebuilding the EDGAR system, where human-written answers set a benchmark for improvement.
