Hugging Face· PlatformCapability change

Accelerate StarCoder with 🤗 Optimum Intel on Xeon: Q8/Q4 and Speculative Decoding

Hugging Face and Intel present optimizations for StarCoder-15B on 4th gen Xeon, achieving over 7x inference acceleration through 8-bit and 4-bit quantization combined with assisted generation (speculative decoding). The work uses SmoothQuant for INT8 quantization and demonstrates significant speedup without accuracy loss on the HumanEval benchmark.

From the source

we show more than 7x inference acceleration of StarCoder-15B model on Intel 4th generation Xeon by integrating 8bit and 4bit quantization with assisted generation.

huggingface.co

Announced

30 Jan 2024 00:00 UTC

Detected

03 Aug 2026 21:12 UTC (+916d after)