# Hugging Face — Accelerating Transformers Fine-Tuning with NVIDIA NeMo AutoModel

- Company: Hugging Face (huggingface.co)
- Announced: 2026-06-24T16:00:13+00:00
- Category: developer-tool-release
- Subject: Platform
- Models affected: NVIDIA Nemotron 3 Ultra 550B A55B, Qwen3-30B-A3B, Nemotron 3 Nano 30B A3B, GPT-OSS, DeepSeek V3
- Source: https://huggingface.co/blog/nvidia/accelerating-fine-tuning-nvidia-nemo-automodel
- Record: https://forck.live/items/1473-accelerating-transformers-fine-tuning-with-nvidia-nemo-automodel

NVIDIA announced NeMo AutoModel, an open library built on Transformers v5 that provides Expert Parallelism, DeepEP fused all-to-all dispatch, and TransformerEngine kernels to accelerate fine-tuning of MoE models, achieving 3.4-3.7x higher training throughput and 29-32% less GPU memory.

## Evidence

Verbatim from https://huggingface.co/blog/nvidia/accelerating-fine-tuning-nvidia-nemo-automodel:

> NVIDIA NeMo AutoModel is an open library part of the NVIDIA NeMo framework for building custom generative AI models at scale. NeMo AutoModel builds cleanly on top of v5, adding Expert Parallelism, DeepEP fused all-to-all dispatch, and TransformerEngine kernels, and it leans on v5's dynamic weight loading to bring those optimizations to a broad and growing set of model families. The payoff is 3.4-3.7x higher training throughput and 29-32% less GPU memory on fine-tuning MoE models than native Transformers v5

---

Record: https://forck.live/items/1473-accelerating-transformers-fine-tuning-with-nvidia-nemo-automodel
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md