From the source
Fireworks and fal released a technical analysis of EAGLE-4, a speculative decoding drafter, alongside a public benchmark dataset for creative tasks.
- Speculative decoding uses a lightweight model to draft several tokens that the target model verifies together, increasing computation per read of the model weights and making better use of idle GPU compute.
- The released Qwen 3.8 27B model makes two changes to the drafting path: sliding-window attention bounds the drafter's attention context, and a low-rank candidate head scores the vocabulary in a smaller representation before computing full logits for selected candidates.
- Multiple draft layers, mixture-of-experts support and Latent ReLU are not yet included in the released model, though the authors expect future models to use them.
- The accompanying Prompt Enhancement SpecBench contains 400 examples from 18 datasets across text-to-video, image-to-video, text-to-image and image-to-image, 100 per category.
- EAGLE-4's harness uses a one-time offline calibration on the target GPU type and parallelism to build a cost table of verify latency across batch size, context length and candidate count, widening or deepening the tree while the estimated gain per marginal cost stays above a threshold.
