# Hugging Face — The Technology Behind BLOOM Training

- Company: Hugging Face (huggingface.co)
- Announced: 2022-07-14
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://huggingface.co/blog/bloom-megatron-deepspeed
- Record: https://forck.live/items/2177-the-technology-behind-bloom-training
- Subject: Platform
- Models affected: BLOOM

The technology and engineering behind training the 176B parameter BLOOM model, including hardware, software stack, and parallelization techniques.

## Evidence

Verbatim from https://huggingface.co/blog/bloom-megatron-deepspeed:

> This article aims to change this by shedding some light on the technology and engineering behind training such models both in terms of hardware and software on the example of the 176B parameter language model BLOOM.

---

Record: https://forck.live/items/2177-the-technology-behind-bloom-training
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md