# Hugging Face — Faster Text Generation with Self-Speculative Decoding

- Company: Hugging Face (huggingface.co)
- Announced: 2024-11-20T00:00:00+00:00
- Category: research-paper
- Subject: Platform
- Models affected: facebook/layerskip-llama2-7B, facebook/layerskip-llama3.2-1B, facebook/layerskip-llama3-8B, meta-llama/Meta-Llama-3-8B, meta-llama/Llama-3.2-1B, meta-llama/Llama-3.2-3B, Llama2 70B, Code Llama
- Source: https://huggingface.co/blog/layerskip
- Record: https://forck.live/items/1796-faster-text-generation-with-self-speculative-decoding

This blog post introduces self-speculative decoding, a technique that uses early layers of a large language model for drafting tokens and later layers for verification, achieving faster text generation and memory savings. It provides implementation details using the transformers library and benchmarks comparing with traditional speculative decoding.

## Evidence

Verbatim from https://huggingface.co/blog/layerskip:

> Self-speculative decoding, proposed in LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding is a novel approach to text generation. It combines the strengths of speculative decoding with early exiting from a large language model (LLM).

---

Record: https://forck.live/items/1796-faster-text-generation-with-self-speculative-decoding
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md