DeepSeek open-sources the inference engine behind its API. And Huawei gets equal billing with Nvidia.

DeepSeek has open-sourced DeepInfer, the inference engine it uses to serve its models in production, a step it said last year was not yet practical.

• The framework lets developers run DeepSeek's V4.1 Flash model on their own infrastructure with many of the optimizations behind the company's API, and support for future DeepSeek models is planned.
• DeepSeek says it is sharing its production stack to help the community lower compute costs, reduce energy use and make AI more affordable.
• Released under the Apache 2.0 license, DeepInfer bundles optimized computing kernels and communication libraries, including the previously open-sourced FlashMLA, DeepGEMM and DeepEP, plus multi-chip distribution, speculative decoding, asynchronous scheduling and structured outputs.
• A companion library, deepseek-recipe, handles how API requests become model inputs and how outputs become responses, and is available for Rust and Python.
• The public software currently supports offline inference with the core features needed for serving at scale, but KV cache management, cross-machine stage separation and replica load balancing are still missing.
• Supported hardware is Nvidia's SM100 architecture, used in Blackwell data center GPUs, and Huawei's Ascend 950 NPU series, while older Nvidia architectures such as Hopper are absent.