From the source
To achieve fast per-token inference throughput for the 176B-parameter BLOOM model using DeepSpeed and Accelerate, including benchmarks on 8x80GB A100 GPUs.
Incredibly Fast BLOOM Inference with DeepSpeed and Accelerate
From the source
This article shows how to get an incredibly fast per token throughput when generating with the 176B parameter BLOOM model.
huggingface.co
