From the source

Benchmarking Text Generation Inference

Developer tool

Hugging Face introduces the TGI Benchmarking tool, a tool to profile Text Generation Inference deployments beyond simple throughput.

From the source
In this blog we will be exploring Text Generation Inference’s (TGI) little brother, the TGI Benchmarking tool.
huggingface.co

Other stories

Hugging Face· 16 OctNew model

Reflection's 501B MoE model is open under Apache 2.0

Hugging Face· 15 OctModel update

IBM's Granite Switch 4.1 models combine chat, decisions, and speech in one endpoint

Hugging Face· 15 OctModel update

Open-weight models gain vision, audio, and thinking mode

Hugging Face· 15 OctOpen-weight release

Meta releases open-weight multimodal MoE model with 816B parameters

Hugging Face· 15 OctOpen-weight release

Open image model adds layout control via bounding boxes

A user creates six custom models via ML Intern in days