From the source
Hugging Face and Intel have adapted and optimized assisted decoding (speculative sampling) for Intel Gaudi processors, now integrated into Optimum Habana to accelerate text generation.
Faster assisted generation support for Intel Gaudi
Story of the weekCapability change
From the source
We adapted and optimized it for Intel Gaudi, which delivers similar performance as Nvidia H100 GPUs as shown in a previous post, while its price is in the same ballpark as Nvidia A100 80GB GPUs. This work is now part of Optimum Habana, which extends various Hugging Face libraries like Transformers and Diffusers so that your AI workflows are fully optimized for Intel Gaudi processors.
