Hugging Face and AMD announce out-of-the-box support for running Transformers models on AMD Instinct GPUs without code changes, including Flash Attention v2, Paged Attention, DeepSpeed, GPTQ, Optimum-Benchmark, and ONNX Runtime integration.
Performance benchmarks show the MI250 GPU delivering over 2.33x more decode throughput and half the prefill latency compared to an A100 card.
From the source
We now support all Transformers models and tasks on AMD Instinct GPUs. And our collaboration is not stopping here, as we explore out-of-the-box support for diffusers models, and other libraries as well as other AMD GPUs.
