Baidu· ERNIEDeveloper tool

FastDeploy 2.0: A Large-Scale Model Inference and Deployment Toolkit with Native Support for ERNIE 4.5

Baidu announced FastDeploy 2.0, an end-to-end toolkit for efficient deployment and high-performance inference of large models, with native support for the ERNIE 4.5 family. It features high-throughput inference using Expert Parallel and Prefill-Decode Disaggregation, near-lossless 2-bit quantization enabling trillion-parameter models on a single GPU, and optimizations for heterogeneous hardware. Benchmark results show significant throughput gains over vLLM for ERNIE and other models.

From the source

FastDeploy 2.0, built on the PaddlePaddle framework , addresses this demand by offering an end-to-end toolkit for efficient deployment and high-performance inference of large models.

ernie.baidu.com

Announced

14 Aug 2025 00:00 UTC

Detected