From the source
Baseten opened the Voice Agent Stack and Baseten Realtime Voice to private preview through a waitlist on November 16, 2026, reporting roughly one-second median turn latency across up to 56 concurrent sessions in its test setup.
- The stack runs on Baseten Dedicated Inference, placing the orchestrator and speech models in one cloud region over an in-cluster network, because Baseten says a remote orchestrator can add hundreds of milliseconds of network round trips per turn even when all speech components are hosted on its platform.
- Realtime Voice is the reference implementation, running a cascade of Baseten-optimized open-weight models with a custom Pipecat-based orchestrator, and an optional delegation design keeps a co-located talker model on conversation while a stronger external thinker model handles reasoning, web search and tool calls asynchronously.
- Customers can pick the speech-to-text, language and text-to-speech models independently, combine dedicated components with third-party APIs, and switch a session's language model, voice or transcription route without redeployment.
- The API matches the core
OpenAI Realtime event protocol, but OpenAI's Python SDK cannot connect directly because it appends /realtime to its base URL, a path Baseten's endpoint does not serve, and browser clients need a backend to hold the API key and relay the socket.
- Baseten reported median time to first audio rising from 0.94 seconds with one concurrent call to 1.14 seconds at 56, and estimated $0.012 per audio minute at 32 concurrent sessions and $0.008 per call-minute at 56 sessions under its stated utilization assumptions.
- An independent Coval leaderboard timestamped 17:30 UTC on November 16, 2026 ranked Baseten-hosted Qwen3 ASR 1.7b second by median Time to Final Segment at 33 milliseconds, with a 2.1% word error rate over 456 samples.