【ローカルLLM】Qwen3.8-27Bの推論性能をテストする（WSL2 + Ollama + RTX 5070 Ti）A hands-on benchmark of Qwen3.8-27B running locally via Ollama on WSL2 with an…

Running large language models on local hardware has become increasingly practical, and this hands-on report walks through benchmarking Qwen3.8-27B, a newly released model, on a consumer-grade setup that combines WSL2, Ollama, and an NVIDIA RTX 5070 Ti. For developers and enthusiasts who want reasoning-capable models without depending on cloud APIs, understanding how a roughly 27-billion-parameter model behaves on a single desktop GPU is a useful data point, both for privacy-sensitive workflows and for controlling recurring inference costs.

The test environment is built on Windows Subsystem for Linux 2, which lets a Linux userland run on top of Windows while passing GPU acceleration through to CUDA workloads. On that foundation sits Ollama, a widely used runtime that wraps llama.cpp and simplifies pulling, configuring, and serving models through a local HTTP API. This particular combination has become a common path for people who keep Windows as their primary desktop but still want the Linux tooling and driver stack that most inference projects assume by default. It also lowers the barrier for those who do not want to dual-boot or maintain a separate Linux machine.

The model itself is served in the GGUF format using Q4_K_M quantization. GGUF is the container format popularized by the llama.cpp project, and Q4_K_M is a 4-bit mixed-precision scheme that aims to balance memory footprint against output quality. At this level of quantization, a 27B model typically occupies somewhere in the mid-teens of gigabytes, which places it close to the memory ceiling of consumer cards. On a GPU like the RTX 5070 Ti, part of NVIDIA's Blackwell-based RTX 50 series, VRAM headroom is therefore a central concern, and it likely influences how much of the model and its context can stay resident on the GPU rather than spilling to slower system memory.

One notable practical detail stands out. While Hugging Face distributes an official Qwen3.8-27B-GGUF:Q4_K_M build, the author reports a problem with how Think mode output is separated when that specific build runs under Ollama, and consequently chose not to use it for the benchmark. This kind of friction is common in the local-LLM space, where the model's chat template, special tokens, and formatting metadata must line up precisely with the runtime for features to behave as intended. When they do not, reasoning traces and final answers can bleed together, tags can leak into the visible response, or the model may not switch modes correctly.

Think mode refers to the reasoning behavior in which a model generates an intermediate chain of thought before producing its final answer. The Qwen series, developed by Alibaba, has embraced this pattern, and recent Qwen3 releases are known for hybrid designs that can toggle between a deliberate reasoning path and a faster direct response. This mirrors a broader industry shift toward explicit reasoning models, a trend accelerated by systems such as OpenAI's o1 and DeepSeek's R1, where allowing the model to "think" longer tends to improve performance on math, coding, and multi-step logic tasks at the cost of additional tokens and latency. Evaluating reasoning quality locally is meaningful precisely because these extra tokens directly affect throughput and responsiveness on constrained hardware.

For readers approaching this from scratch, a few prerequisite concepts help frame the results. Quantization trades numerical precision for reduced memory use, and heavier compression generally degrades accuracy, so a Q4_K_M build represents a compromise rather than the model's full-quality ceiling. Throughput is usually measured in tokens per second, and for reasoning workloads the length of the hidden thought process can substantially change the effective wait time before an answer appears. Context window size, prompt length, and whether the model fully fits in VRAM all interact to shape the experience.

Taken together, the report reflects the current state of running mid-sized reasoning models at home: capable but sensitive to configuration. The choice to avoid the problematic GGUF build in favor of a working alternative underscores how much of local inference remains about matching the right model file to the right runtime and template. As tooling like Ollama, llama.cpp, and the surrounding ecosystem continues to mature, and as consumer GPUs gain memory and compute, setups such as WSL2 paired with an RTX 5070 Ti appear likely to make locally hosted reasoning models steadily more accessible to individual developers.