From the source
Amazon's post explains how AssemblyAI runs its speech-to-text inference on Amazon EC2 Spot Instances to cut GPU costs while keeping its SLA commitments.
- Speech-to-text inference is computationally intensive, with each audio file triggering a pipeline of deep learning models on
NVIDIA GPUs, which at scale means thousands of GPU instances running continuously.
- Running entirely on On-Demand Instances would be cost-prohibitive, and Spot Instances can be reclaimed with a 2-minute warning, so AssemblyAI needed an architecture that absorbs interruptions gracefully.
- The platform runs on Amazon ECS with capacity providers that mix Spot and On-Demand capacity, spanning g4dn, g5 and g6 GPU instance families and using the price-capacity optimized allocation strategy.
- When a Spot Instance gets an interruption notice, an EventBridge-triggered Lambda returns the in-flight job to the Main Job SQS Queue for another ECS task to pick up, and jobs that exhaust retries go to a Dead Letter SQS Queue.
- AssemblyAI deploys regionally in US West (Oregon) us-west-2 and Europe (London) eu-west-2 to meet data residency rules for customers in healthcare, finance and legal.






