From the source
Hugging Face released LoRA support in TRL's AsyncGRPOTrainer (v1.14), enabling training of LoRA adapters that sync only the adapter weights to vLLM inference servers rather than full model weights.
- The implementation allows trainer and inference to run on separate Hugging Face Jobs using Storage Buckets for adapter synchronization and a proxy server for routing and broadcasting, reducing training time from 3 hours 27 minutes to 53 minutes for 500 steps in a real-world project.






