ローカルLLM向けハードウェアを「容量・帯域・MoE・TTFT」で選ぶThis article explains how to choose hardware for running local LLMs by…

Running large language models locally has moved from a niche experiment to a practical option for developers, researchers, and privacy-conscious organizations. Yet the hardware choices involved are not always intuitive, and buying the most expensive GPU is not necessarily the right move. A recent Zenn blog post argues that anyone selecting hardware for local inference should evaluate it along four distinct axes: memory capacity, memory bandwidth, support for Mixture-of-Experts (MoE) architectures, and time-to-first-token (TTFT). Framing the decision this way helps clarify why two setups with similar raw specifications can deliver very different real-world experiences.

The first axis, memory capacity, determines whether a model fits at all. Model weights, the key-value cache that grows with context length, and runtime overhead all consume memory. Quantization, typically to 8-bit or 4-bit precision using formats such as GGUF or GPTQ, dramatically reduces the footprint and is often the deciding factor in whether a large model runs on consumer hardware. When capacity falls short, part of the model must be offloaded to system RAM and executed on the CPU, which brings the second axis into focus.

Memory bandwidth is frequently the real bottleneck during token generation. Autoregressive decoding reads the entire set of active weights for every token produced, so the speed at which memory can be streamed to the compute units tends to matter more than peak floating-point throughput. This is why high-bandwidth GPU memory, and increasingly unified memory architectures such as Apple Silicon or systems built around fast LPDDR, can outperform expectations. It also explains why CPU offload is slow: system RAM bandwidth is far lower than that of dedicated GPU memory, so pushing layers to the CPU can cause generation speed to drop sharply.

The third axis, MoE support, reflects a shift in how leading models are built. Mixture-of-Experts designs, used in models such as Mixtral and several newer releases, activate only a subset of their parameters for each token. This means a model may hold a very large number of total parameters while only a fraction are computed at any step. The practical consequence is a tension between the two earlier axes: total parameters drive memory capacity requirements, but only the active experts drive per-token compute and bandwidth demand. MoE models can therefore run surprisingly well on systems with ample but slower memory, making CPU offload and unified-memory machines more attractive than they would be for equivalently sized dense models.

The fourth axis, TTFT, captures latency rather than throughput. Time-to-first-token measures how long a user waits before the model begins responding, and it is dominated by the prefill stage, where the entire input prompt is processed in parallel. Unlike generation, prefill is compute-intensive, so raw processing power and prompt length weigh heavily here. Applications with long contexts, such as document analysis or retrieval-augmented generation, feel this cost acutely, and it is a separate consideration from the tokens-per-second figure that describes ongoing output.

Taken together, these axes map onto concrete hardware categories. A discrete GPU with high-bandwidth memory offers the best balance for dense models but is constrained by its fixed capacity. CPU offload setups trade speed for the ability to run larger models cheaply, an approach that appears especially reasonable for MoE architectures. Dedicated accelerators and unified-memory systems occupy a middle ground that is likely to grow more relevant as model designs evolve.

Several adjacent tools make these trade-offs manageable in practice. Inference runtimes such as llama.cpp, Ollama, vLLM, and LM Studio expose settings for quantization level, the number of layers offloaded to the GPU, and context size, letting users tune a system to its bottleneck. The broader industry context reinforces the article's premise: vendors are competing not only on compute but on memory bandwidth and capacity, and the popularity of MoE models is reshaping assumptions about what "large" means for local deployment.

The key takeaway is that no single number captures a machine's suitability for local LLMs. A buyer focused solely on capacity may end up with slow generation, while one focused solely on bandwidth may be unable to load the model they want. By reasoning through capacity, bandwidth, MoE behavior, and TTFT separately, and matching them to the specific models and workloads in mind, users can make more informed decisions and avoid overpaying for specifications that their actual use case never exercises.