［2026年版］最新Open LLMのアーキテクチャ総整理（Kimi K3, GLM-5.2, etc.）A 2026 survey comparing the architectures of leading open LLMs including Kimi…

The pace at which open large language models evolve has turned architecture comparison into a practical necessity rather than an academic exercise. A recent survey published on Qiita sets out to catalogue and contrast the internal designs of the leading open models available in 2026, including Moonshot AI's Kimi K3 and Zhipu's GLM-5.2, with the stated aim of helping practitioners match a model to a given task, hardware profile, and deployment budget.

The article's central premise is that while nearly all modern open LLMs share a common transformer-decoder lineage, the differences that now matter for real-world use live in the details. Since the first wave of Llama-style models, the field has largely converged on a handful of building blocks: rotary position embeddings (RoPE), RMSNorm in place of the original LayerNorm, SwiGLU feed-forward layers, and grouped-query attention (GQA) to shrink the memory footprint of the key-value cache. What separates today's frontier open models, the survey argues, is how they extend or replace these components rather than whether they use a transformer at all.

Mixture-of-Experts (MoE) appears to be the single most consistent trend across the models compared. Rather than activating every parameter for every token, MoE architectures route each token through a small subset of specialized expert networks, allowing total parameter counts to grow into the hundreds of billions while keeping the active, per-token compute comparatively modest. The article notes that design choices within this pattern vary widely, including the number of experts, how many are activated per token, the use of shared or "always-on" experts, and the presence of fine-grained routing. These decisions directly affect inference cost and the amount of memory required to hold the full weight set, which is especially relevant for anyone attempting to run such models locally.

Attention mechanisms form a second axis of comparison. Several models have moved beyond standard GQA toward variants aimed at reducing the key-value cache further, a limiting factor for long-context inference. Multi-head Latent Attention, popularized by DeepSeek, compresses the cache into a lower-dimensional latent space, while sliding-window or local attention, seen in the Gemma and Mistral families, restricts most layers to a bounded context to save memory. The survey also touches on normalization placement and additions such as QK-normalization, which are small structural tweaks that reportedly improve training stability at scale.

Context length and positional handling receive their own treatment. As models push toward very long context windows, techniques for extending or interpolating RoPE, and in some cases hybrid schemes that combine positional encodings with layers that omit them, are described as increasingly common. The article frames these as trade-offs rather than universal improvements, since aggressive context extension can affect quality on shorter inputs and increase compute at inference time.

For readers focused on local deployment, the practical value lies in connecting these architectural facts to operational constraints. Whether a model uses MoE, how large its active parameter count is, and how efficiently it manages the key-value cache all influence how well it will run under quantization on consumer or prosumer hardware. The broader ecosystem is relevant here too: runtimes such as llama.cpp, vLLM, and SGLang, along with quantization formats like GGUF and AWQ, determine how quickly a newly released architecture becomes usable outside of a data center, and support for novel attention or routing schemes often lags the model release itself.

The survey sits within a wider industry context in which the gap between open and proprietary systems has continued to narrow, driven in significant part by Chinese labs releasing capable weights under permissive terms. Readers should treat any single comparison as a snapshot; naming conventions, benchmark claims, and licensing details shift quickly, and the specific figures cited for models like Kimi K3 and GLM-5.2 are best verified against each project's own technical reports. As a structured overview, however, the article is likely to be most useful as a mental map of the design space, clarifying which architectural levers exist and why a given model may be better suited to one workload than another.