実際どれだけ速くなる？ Gemma-4のMTPを、普段使いのPCでllama.cppで検証してみたThis article benchmarks Gemma-4's multi-token prediction (MTP) running on…

Multi-token prediction, or MTP, has become one of the more discussed techniques for accelerating local large language model inference, and a new hands-on write-up sets out to measure how much of its promised speed-up actually appears on an ordinary home computer. Published on Zenn as the second installment in a two-part series, the article moves from theory to practice by running Gemma-4's MTP implementation on llama.cpp, the widely used C/C++ inference engine, and reporting real measurement data rather than relying on vendor claims.

The piece is explicitly framed as a continuation of an earlier "concept" article. That first part described MTP as a kind of low-risk wager: a bet where you lose almost nothing when you miss but gain when you hit. The reasoning is that predicting several tokens ahead costs little extra if the guesses are wrong, yet saves meaningful time when they are correct. The earlier article also noted that two prominent open-model families, Qwen and Gemma, implement this bet in markedly different ways. Having established the intuition, the author turns to the natural follow-up question: how much faster does it really get, and is the bet actually paying off?

To understand why this matters, it helps to place MTP alongside speculative decoding, the broader family of methods it belongs to. In conventional autoregressive generation, a model produces one token at a time, and each step requires a full forward pass. Speculative decoding tries to break that bottleneck by cheaply drafting multiple candidate tokens and then verifying them in a single pass with the main model. When the draft is accepted, several tokens are emitted for roughly the cost of one verification step; when it is rejected, the system falls back with limited penalty. MTP embeds this idea more tightly into the model itself, using additional prediction heads to propose upcoming tokens instead of relying on a separate, smaller draft model.

llama.cpp is a natural testbed for this kind of experiment because it has become the default way many enthusiasts run open models locally, supporting quantized weights and CPU or modest GPU execution. The tool has steadily added support for speculative decoding and draft models, so evaluating MTP within it reflects how everyday users are most likely to encounter the feature. By focusing on a typical consumer PC environment rather than data-center hardware, the article targets the audience most sensitive to inference latency, where even a modest throughput gain can change whether a model feels usable interactively.

The core of the verification, as described, is empirical: the author runs Gemma-4 with MTP enabled on llama.cpp and records how token generation speed changes compared with standard decoding. Because real-world speed-ups from speculative techniques depend heavily on the acceptance rate of the drafted tokens, results are likely to vary with the prompt, the sampling settings, and the specific hardware. Tasks with more predictable output, such as structured or repetitive text, tend to benefit more, while highly varied generation may see smaller gains. The article's value lies in grounding these expectations in measured numbers on accessible hardware.

For readers weighing whether to enable such features, a few prerequisites and caveats are worth keeping in mind. The realized benefit is not guaranteed; overhead from managing draft tokens can erode gains if acceptance is low, and memory or compute constraints on a home machine may limit how aggressively the technique can be applied. The comparison between Gemma's and Qwen's approaches also suggests there is no single "correct" MTP design, and the trade-offs each makes may suit different workloads.

As open models increasingly ship with built-in acceleration mechanisms, practical benchmarks like this one help clarify the gap