LLM推論を最大2倍高速化するEAGLE 3.1 — attention driftを克服した最新スペキュラティブデコーディング EAGLE 3.1, released May 26 2026, addresses 'attention drift' in speculative decoding and a…

Speculative decoding just cleared another significant bar. EAGLE 3.1, released on May 26 2026, resolves a structural flaw called 'attention drift' that had been quietly undermining draft-model accuracy in long-generation scenarios, and the results are striking: vLLM's official benchmarks record a 2.03× throughput improvement over EAGLE-3 on the Kimi K2.6 model under single-request conditions.

For those unfamiliar with the technique, speculative decoding works by pairing a large target model with a smaller, faster draft model. The draft model proposes several tokens ahead, and the target model verifies them in a single parallel pass — accepting correct guesses and discarding wrong ones. The net effect is higher throughput without changing the output distribution. EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency) has been one of the leading implementations of this approach, feeding the target model's internal feature representations into the draft model to achieve unusually high token acceptance rates.

The 'attention drift' problem that EAGLE 3.1 targets is subtle but cumulative. As generation length grows, the attention context that the draft model uses to predict upcoming tokens gradually diverges from the context the base model is actually building. This misalignment erodes acceptance rates over time, making the speedup less reliable for longer outputs — precisely the kind of outputs that matter most in real-world usage. EAGLE 3.1 addresses this at the architectural level, likely by enforcing tighter KV-cache consistency between the draft and target model attention states, though the full technical details are available in the accompanying paper and repository.

The practical significance of a 2× throughput gain is hard to overstate for local LLM deployments. Running a 70B or larger model on-premises is already expensive in terms of GPU memory and compute; halving the effective latency per token can either double serving capacity for the same hardware budget or dramatically reduce the cost per query. vLLM integration is reportedly already in progress, which means production users could benefit without waiting for a major framework release cycle.

A few caveats are worth noting. The 2.03× figure comes from a single-request benchmark, and speculative decoding is well-known to lose efficiency as batch sizes increase — the parallel verification step becomes less of an advantage when the GPU is already saturated with concurrent requests. High-throughput, heavily batched inference deployments will need independent benchmarking before drawing conclusions. That said, for the growing class of single-user or low-concurrency local inference servers, the gains look genuinely compelling.

EAGLE is developed by a research group based at Peking University, and the series has consistently outperformed competing approaches — Medusa, Hydra, Lookahead Decoding, and others — in published comparisons. EAGLE 3.1 reinforces that the speculative decoding space still has meaningful headroom for algorithmic improvement, independent of hardware advances. As models continue to scale and inference costs become a dominant concern, techniques like this may end up mattering as much as chip-level efficiency gains.