Hugging Face announced that they have achieved a 100x speedup in transformer inference for their Accelerated Inference API customers through a combination of high-level optimizations using Hugging Face libraries and low-level hardware-specific compilation techniques.
From the source
This 100x performance gain and built-in scalability is why subscribers of our hosted Accelerated Inference API chose to build their NLP features on top of it.
