# Hugging Face — Faster Assisted Generation with Dynamic Speculation

- Company: Hugging Face (huggingface.co)
- Announced: 2024-10-08T00:00:00+00:00
- Category: new-model
- Subject: Platform
- Source: https://huggingface.co/blog/dynamic_speculation_lookahead
- Record: https://forck.live/items/1819-faster-assisted-generation-with-dynamic-speculation

Hugging Face and Intel labs announce dynamic speculative decoding, a new method for accelerating text generation up to 2.7x, which becomes the default in Transformers version 4.45.0. The method adjusts the number of draft tokens based on confidence, outperforming the heuristic approach.

## Evidence

Verbatim from https://huggingface.co/blog/dynamic_speculation_lookahead:

> dynamic speculative decoding—a novel method developed by Intel labs and Hugging Face that accelerates text generation by up to 2.7x, depending on the task. This method is the default operational mode for assisted generation starting from Transformers🤗 release 4.45.0

---

Record: https://forck.live/items/1819-faster-assisted-generation-with-dynamic-speculation
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md