【ローカルLLM】Qwen3.8-27BをデュアルGPUで高速化するThis article explores running Qwen3.8-27B locally on a dual-GPU setup to…

Running large language models locally continues to attract developers who want privacy, offline capability, and control over their inference stack. A recent hands-on write-up from the qiita-llm blog examines how to push Qwen3.8-27B beyond the limits of a single consumer graphics card by splitting the workload across two GPUs, and why doing so matters for anyone trying to run mid-to-large models on desktop hardware.

The article builds directly on an earlier experiment in which the author ran Qwen3.8-27B on a single RTX 5070 Ti, a card equipped with 16 GB of VRAM, using Ollama as the runtime. That first test established baseline inference performance but ran into a fundamental constraint: the official Ollama Q4 quantized version of the 27B model does not fit comfortably within 16 GB of VRAM alone. When a model's weights exceed available video memory, the runtime is forced to offload layers to system RAM and the CPU, which typically causes a sharp drop in throughput and less predictable behavior. That bottleneck is the problem the dual-GPU configuration is meant to solve.

To understand the stakes, it helps to recall how quantization and memory budgeting interact. Q4 refers to roughly four-bit quantization, a compression technique that shrinks model weights so that large models can run on modest hardware with only a modest quality trade-off. Even so, a 27-billion-parameter model at four-bit precision still needs well over 16 GB once weights, the key-value cache, and context buffers are accounted for. This is a common wall for owners of single 16 GB cards, and it explains why many local-LLM enthusiasts either drop to smaller models, accept partial CPU offloading, or add a second GPU.

The dual-GPU route described in the piece addresses the shortfall by pooling video memory across two cards so more of the model can remain resident on the GPUs rather than spilling to the CPU. Ollama, which is built on top of the llama.cpp inference engine, supports distributing model layers across multiple GPUs, and the underlying libraries can split a model so that different layers reside on different devices. The stated result is that Qwen3.8-27B runs faster and more stably in the dual-GPU setup than it did when the single RTX 5070 Ti had to lean on system memory. The gains appear to come primarily from keeping the full working set in fast video memory and avoiding the latency penalty of CPU offloading.

It is worth noting some context around the tools involved. Ollama has become a popular entry point for local inference because it packages model downloads, quantization variants, and a simple serving interface into one command-line experience, lowering the barrier for running open-weight models on a personal machine. The Qwen family, developed by Alibaba, is a widely used series of open-weight models that ships in several parameter sizes, making it a frequent choice for people benchmarking consumer GPUs. The RTX 5070 Ti sits in the mainstream-to-upper tier of consumer cards, and pairing two of them is a comparatively accessible way to reach the memory capacity that a single flagship card might otherwise provide.

Readers considering a similar build should keep a few practical factors in mind that generally accompany multi-GPU inference. Splitting a model across cards introduces communication overhead between devices, so the speedup is not always linear and depends on how layers are partitioned, the PCIe bandwidth available, and how the runtime schedules work. Power draw, cooling, motherboard lane allocation, and driver support are also part of the equation when installing two cards in one system. These considerations are typical trade-offs rather than flaws specific to this setup, but they influence whether the reported stability and speed improvements translate to other configurations.

Overall, the article serves as a targeted case study rather than a universal benchmark: it documents one path for running a 27B-class model that overflows a single 16 GB card, using widely available tooling. For the growing community experimenting with local LLMs on prosumer hardware, it reinforces a recurring lesson that VRAM capacity, not raw compute alone, is often the deciding factor, and that adding a second GPU can be a pragmatic way to keep large quantized models running entirely on the graphics subsystem.