拡散言語モデルとは？LLMが2.42倍速くなる新しい生成の仕組みDiffusion language models generate multiple tokens in parallel rather than one…

Diffusion language models are emerging as an alternative to the autoregressive designs that power most of today's large language models, and they promise a meaningful change in how text is generated. Rather than producing one token at a time, these models generate or refine many tokens in parallel, and recent reports suggest this can deliver up to a 2.42x speedup in inference compared with conventional autoregressive systems. For anyone running models locally, where compute and latency are hard constraints, that kind of gain is significant.

To understand why the approach differs, consider how a standard autoregressive model such as those in the GPT or Llama families works. It predicts text left to right, choosing the next token based on everything generated so far, then feeding that token back in to predict the following one. This sequential dependency is what makes the output coherent, but it also creates a fundamental bottleneck: generating a response of N tokens requires roughly N forward passes through the network. Longer outputs mean proportionally more waiting, and no amount of parallel hardware fully removes the step-by-step nature of the process.

Diffusion language models borrow their core idea from the diffusion models that transformed image generation. Those image systems work by gradually adding noise to data during training and then learning to reverse the process, denoising a random field into a coherent picture over a series of steps. Text diffusion adapts this to discrete tokens. A common formulation is masked diffusion: the model starts from a sequence that is largely or entirely masked and iteratively fills in tokens, refining the whole sequence across a fixed number of denoising steps. Because each step can update many positions at once, and because the number of steps can be smaller than the sequence length, the model can in principle produce long passages with far fewer sequential operations.

The speed advantage comes from this decoupling of output length from the number of network passes. Where an autoregressive model needs one pass per token, a diffusion model might reach a usable result in a few dozen steps regardless of how many tokens it emits. The tradeoff is typically quality against step count: fewer denoising iterations run faster but can leave errors or less fluent text, so implementations tune the number of steps to balance latency and output quality. Reported figures like 2.42x are best read as results under particular benchmarks and settings rather than universal guarantees.

There are also architectural differences worth noting. Autoregressive models use causal attention, where each position sees only earlier tokens, which enables the key-value caching that makes their incremental decoding efficient. Diffusion language models generally use bidirectional attention, letting every position attend to the full context, which can improve global consistency but changes the optimization picture and makes the familiar KV-cache tricks less directly applicable.

The idea is not entirely new. Non-autoregressive generation was explored for years in machine translation, where researchers sought to emit whole sentences at once rather than word by word. What has changed is scale and viability. In 2025 the space gained visible momentum: Inception Labs introduced Mercury, a commercial diffusion-based model aimed at fast code and text generation, and Google DeepMind previewed Gemini Diffusion. On the open side, research models such as LLaDA and Dream 7B demonstrated that diffusion approaches can reach competitive quality at the several-billion-parameter scale, giving the local and open-source community concrete systems to experiment with.

For local LLM users specifically, the appeal is practical. Faster generation at a given hardware budget could make interactive applications more responsive and reduce the cost of long outputs. It remains to be seen how broadly diffusion models will match autoregressive systems across reasoning, instruction following, and the tooling ecosystems currently built around token-by-token decoding. Inference engines, quantization methods, and serving frameworks are heavily optimized for the autoregressive case, so adoption is likely to depend as much on software support as on underlying model quality. Still, the parallel-generation paradigm represents one of the more concrete architectural challenges to the autoregressive default, and its progress is worth watching closely as the underlying research and available implementations continue to mature.