Qwen3.8-27BはMoEではなかった — ローカル音声対話AIへの採用を30回計測して見送るまでThe author evaluated replacing Qwen3.6-35B-A3B (MoE) with Qwen3.8-27B in a…

Running a capable large language model entirely on local hardware has become an attainable goal for developers who prioritize privacy, predictable latency, and freedom from per-token cloud fees. A recent blog post walks through one such effort: a developer maintaining a fully local voice-dialogue assistant tried to upgrade its underlying model, then decided against the change after thirty rounds of measurement. The takeaway is a reminder that a model's headline parameter count can hide the architectural details that actually determine real-world performance.

The setup runs on an AMD Ryzen AI MAX+ 395, an APU whose integrated GPU is identified as gfx1151, paired with a Qwen3.6-35B-A3B model. That "A3B" suffix is the crux of the story. It denotes a Mixture-of-Experts (MoE) design in which, although the model holds roughly 35 billion parameters in total, only about 3 billion are activated for any given token. A routing network selects a small subset of "experts" per step, so the compute and memory traffic per token stay low even though the model's overall capacity is large. For an interactive use case like voice dialogue, where the language model sits between speech recognition and speech synthesis, that efficiency translates directly into a shorter response delay.

Attracted by the newer Qwen3.8-27B, the author considered swapping it in, apparently on the assumption that a 27B model might follow the same efficient pattern. On inspection, however, the 27B turned out to be a dense model rather than an MoE one. In a dense architecture, every parameter participates in generating every token, so all 27 billion weights are engaged at each step rather than a roughly 3-billion active slice.

That distinction matters a great deal on this class of hardware. Systems built around unified-memory APUs are frequently bound by memory bandwidth, because each generated token requires reading the active weights from memory. A dense 27B model must move far more data per token than a 35B MoE that activates only about 3B parameters. As a result, even though 27B is a smaller number than 35B, the dense model is likely to run considerably slower in tokens per second on the same machine. The author's benchmarking, described as thirty runs comparing speed and quality, appears to have confirmed that the trade-off did not favor the switch, and the upgrade was shelved.

The episode illustrates a broader industry trend. Over the past few years, MoE designs have gained popularity precisely because they decouple model capacity from per-token compute, allowing large total parameter counts while keeping inference costs manageable. Models across several families have adopted the approach, and the Qwen line from Alibaba has offered both dense and MoE variants at various sizes, which is exactly why matching a naming convention to an architecture can be error-prone. A number in a model name indicates total parameters; whether those parameters are dense or sparsely activated is a separate fact that