From DeepSpeed to FSDP and Back Again with Hugging Face Accelerate
Capability change
Hugging Face Accelerate introduces a precision-related change and a concept guide to harmonize FSDP and DeepSpeed backends, enabling users to seamlessly switch between them.
The change, included in the 0.30.0 release, allows FSDP to operate in a mixed-precision mode similar to DeepSpeed or a low-precision mode for memory-constrained scenarios.
