From the source
Hugging Face announces multi-backend support for Text Generation Inference (TGI), allowing integration with vLLM, TensorRT-LLM, llama.cpp, AWS Neuron, and Google TPU backends through a unified frontend layer.
Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference
Capability change
From the source
we are excited to introduce the concept of TGI Backends. This new architecture gives the flexibility to integrate with any of the solutions above through TGI as a single unified frontend layer.
huggingface.co
