From the source
Baseten says it optimized pyannote's Community-1 and Precision-2 speaker diarization models, cutting long-audio latency by up to 9.6x and raising throughput by 3.2x.
- The company says the work answers a production problem: high-quality diarization models are slow and costly to run, while cheaper ones are not good enough for voice products.
- Baseten combined quality-aware quantization, index-based clustering and scheduling changes, including fusing the pipeline into a single forward pass and cutting CPU-to-GPU memory transfers.
- The indexing strategy lets a single GPU diarize 20 hours of audio in two minutes, a job Baseten says the unoptimized pipeline cannot finish in one request.
- Mixed precisions across the pipeline lifted Precision-2 throughput by 3.2x with a 0.5% absolute DER penalty, while the clustering loop runs in float64 on the GPU.
- On VoxConverse, Baseten reports 11.1% DER for Community-1 and 8.8% for Precision-2, which it calls state-of-the-art for models of their size and speed.
- Baseten says it will soon add voiceprint enrollment and cross-session speaker identification to its diarization deployments.





