# Hugging Face — From PyTorch DDP to Accelerate to Trainer, mastery of distributed training with ease

- Company: Hugging Face (huggingface.co)
- Announced: 2022-10-21
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://huggingface.co/blog/pytorch-ddp-accelerate-transformers
- Record: https://forck.live/items/2141-from-pytorch-ddp-to-accelerate-to-trainer-mastery-of-distributed-training-with
- Subject: Platform

To use PyTorch DDP, Hugging Face Accelerate, and Hugging Face Trainer for distributed training on multiple GPUs.

## Evidence

Verbatim from https://huggingface.co/blog/pytorch-ddp-accelerate-transformers:

> This tutorial assumes you have a basic understanding of PyTorch and how to train a simple model. It will showcase training on multiple GPUs through a process called Distributed Data Parallelism (DDP) through three different levels of increasing abstraction

---

Record: https://forck.live/items/2141-from-pytorch-ddp-to-accelerate-to-trainer-mastery-of-distributed-training-with
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md