Accelerate ND-Parallel: A guide to Efficient Multi-GPU Training
Capability change
Hugging Face announces integration of ND-Parallel (multi-dimensional parallelism) into Accelerate and Axolotl, allowing users to easily combine data, tensor, context, and fully sharded data parallelism strategies for efficient multi-GPU training.
