Hugging Face· PlatformDeveloper tool

Improving Hugging Face Training Efficiency Through Packing with Flash Attention 2

Hugging Face announces a new feature in Transformers and TRL libraries that allows packing of training examples without padding, compatible with Flash Attention 2, using a new DataCollatorWithFlattening and a padding_free flag. It reports up to 2x throughput improvement and up to 20% peak memory reduction on certain datasets, with no impact on convergence.

From the source

Hugging Face Transformers now addresses this with a new feature that maintains boundary awareness during packing, alongside the introduction of a new data collator, DataCollatorWithFlattening.

huggingface.co

Announced

21 Aug 2024 00:00 UTC

Detected

03 Aug 2026 21:12 UTC (+712d after)