From the source
How presigned URLs, async pipelines, and defensive fallback made Vault uploads faster and more reliable at scale.
Over the past few months, Vault has become one of Harvey’s most heavily used surfaces as legal teams increasingly rely on it as their central system for AI-powered file work.
Weekly upload volume climbed from 2.2 million files per week in mid-January to 15 million files per week in May, while Vault crossed 200 million active files this week.
The first 100 million took roughly two years; the second 100 million took just two months.
After working to scale Vault’s file upload and management last fall, this recent growth in usage made us rethink how we can scale these capabilities even further.
That forced us back to the central architecture question behind large-scale uploads: should Harvey’s backend continue carrying every file byte, or should it coordinate uploads while storage handles the transfer?
Presigned URL uploads are the gold-standard answer to the classic file-upload system design question: keep your application server on the control plane, and keep file bytes on the data plane.
…
