# SiliconANGLE — DeepSeek open-sources DeepInfer, the inference framework behind its production services

- Company: SiliconANGLE (siliconangle.com)
- Announced: 2026-10-20T23:11:43+00:00
- Category: open-weight-release
- Coverage: not counted
- Announcement: yes
- Group: models
- Source: https://siliconangle.com/2026/10/20/deepseek-open-sources-deepinfer-the-inference-framework-behind-its-production-services/
- Record: https://forck.live/items/25513-deepseek-open-sources-deepinfer-the-inference-framework-behind-its-production
- Subject: Reporting
- Open weights: yes
- Models affected: DeepSeek-V4.1-Flash
- License: Apache 2.0
- Pricing: revenue at off-peak pricing works out to $38 per graphics processing unit per hour

DeepSeek has open-sourced DeepInfer, the inference framework behind its own production services, under the Apache 2.0 license on GitHub. The company calls reaching a massive user base at a reasonable price "the core problem an inference framework has to solve," and says one cluster now serves 155 trillion tokens a day, with revenue at off-peak pricing working out to $38 per GPU per hour. DeepInfer supports only DeepSeek-V4.1-Flash for now, loading the updated 1017 version by default, and runs on Nvidia's SM100 Blackwell GPUs and Huawei's Ascend 950 neural processing units; Hopper and other older Nvidia architectures are not on the list. Two features that keep outputs identical from run to run work only on Nvidia hardware, and all published benchmarks, which are DeepSeek's own, ran on Nvidia chips. Speculative decoding runs through DSpark, where a draft model proposes five tokens at a time and a confidence head predicts which the main model will accept, trimming draft tokens checked per step by about 30% and, DeepSeek estimated, adding 4% to 7% throughput at production load. DeepSeek has not released its key-value cache management and load-balancing code, saying the two are tightly coupled to internal infrastructure, and splitting prefill and decode across separate machines is left to operators.

## Evidence

Verbatim from https://siliconangle.com/2026/10/20/deepseek-open-sources-deepinfer-the-inference-framework-behind-its-production-services/:

> DeepSeek has not yet released its key-value cache management and load-balancing code, saying the two are tightly coupled to internal infrastructure. A blog post the company published on Zhihu lays out how both work.

---

Record: https://forck.live/items/25513-deepseek-open-sources-deepinfer-the-inference-framework-behind-its-production
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md