From the source
From the source
NVIDIA announced NeMo AutoModel, an open library built on Transformers v5 that provides Expert Parallelism, DeepEP fused all-to-all dispatch, and TransformerEngine kernels to accelerate fine-tuning of MoE models, achieving 3.4-3.7x higher training throughput and 29-32% less GPU memory.
From the source
NVIDIA NeMo AutoModel is an open library part of the NVIDIA NeMo framework for building custom generative AI models at scale. NeMo AutoModel builds cleanly on top of v5, adding Expert Parallelism, DeepEP fused all-to-all dispatch, and TransformerEngine kernels, and it leans on v5's dynamic weight loading to bring those optimizations to a broad and growing set of model families. The payoff is 3.4-3.7x higher training throughput and 29-32% less GPU memory on fine-tuning MoE models than native Transformers v5
huggingface.co
