# Together AI — Serving DeepSeek-V4: why million-token context is an inference systems problem

- Company: Together AI (together.ai)
- Announced: 2026-05-11T00:00:00+00:00
- Category: model-update
- Subject: Inference platform
- Models affected: DeepSeek-V4
- Context window: 1M-token context window
- Source: https://www.together.ai/blog/serving-deepseek-v4-why-million-token-context-is-an-inference-systems-problem
- Record: https://forck.live/items/2349-serving-deepseek-v4-why-million-token-context-is-an-inference-systems-problem

Together AI discusses serving implications of DeepSeek-V4, focusing on its 1M-token context window and architectural changes with compressed sparse attention mechanisms. They describe how V4's token-axis KV cache compression improves serving capacity but requires sophisticated memory management across multiple cache types (CSA, HCA, SWA). The post shares early bring-up results on NVIDIA HGX B200 hardware, showing increased KV-cache capacity from 1.2M to 3.7M tokens on a single node through cache policy optimizations.

## Evidence

Verbatim from https://www.together.ai/blog/serving-deepseek-v4-why-million-token-context-is-an-inference-systems-problem:

> V4's architecture creates the opportunity for long-context efficiency, but the realized capacity depends on how the inference engine stores, recomputes, and evicts the different cache types.

---

Record: https://forck.live/items/2349-serving-deepseek-v4-why-million-token-context-is-an-inference-systems-problem
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md