Together AI· Inference platformInfrastructure

Cache-aware prefill–decode disaggregation (CPD) for up to 40% faster long-context LLM serving

Together AI announces cache-aware prefill–decode disaggregation (CPD), a serving architecture that separates cold and warm workloads by cache hit rate to improve throughput and reduce time-to-first-token for long-context inference.

From the source

At Together AI, we built cache-aware prefill–decode disaggregation (CPD), a serving architecture that purposely separates cold and warm workloads by cache hit rate, resulting in fast context reuse.

together.ai

Announced

04 Mar 2026 00:00 UTC

Detected

03 Aug 2026 21:12 UTC (+152d after)