RTX 4090 × 4枚で Qwen3.6-27B ファインチューン「Fable-Fusion-711」を vLLM (Docker) で動かす — ハイブリッド構成の KV 設計までThis article walks through running the Qwen3.6-27B community fine-tune…

Running large language models on local hardware remains a popular pursuit for developers who value data control, predictable cost, and the freedom to customize, and a recent Qiita post adds a detailed, hands-on example to that space. The article explains how to serve "Fable-Fusion-711," a community fine-tune of Qwen3.6-27B, across four NVIDIA RTX 4090 GPUs using vLLM inside a Docker container, with particular attention to KV cache design for hybrid configurations.

The model at the center of the guide is Qwen3.6-27B-Fable-Fusion-711, released by the community contributor DavidAU. According to the article, the fine-tune posts a score of 0.711 on ARC-C — the harder "Challenge" split of the AI2 Reasoning Challenge, from which the "711" in the name appears to derive — and surpasses the base Qwen3.6-27B on six of seven benchmarks the author cites. The base model belongs to Alibaba's Qwen family, a widely used series of open-weight models, and the "Fable" naming appears to reflect an orientation toward creative and narrative writing.

The choice of four RTX 4090 cards is significant because each provides 24 GB of VRAM, for a combined 96 GB. A 27B-parameter model stored in 16-bit precision consumes roughly 54 GB for weights alone, leaving the remainder for the KV cache, activations, and runtime overhead. Because the RTX 4090 does not support NVLink, the GPUs communicate over PCIe, which makes the way work is partitioned across cards an important consideration for throughput. vLLM addresses multi-GPU execution primarily through tensor parallelism, which splits individual layers across devices so that a model too large for one card can run across several.

vLLM itself is an open-source inference and serving engine designed for high throughput. Its signature technique, PagedAttention, manages the KV cache in fixed-size blocks that need not be contiguous in memory, an approach modeled on virtual memory paging in operating systems. This reduces fragmentation and allows more requests to be batched together, which matters directly when VRAM is the binding constraint. Packaging the deployment in Docker, as the article does, adds reproducibility by pinning the runtime, drivers, and dependencies into a portable image.

The KV cache is worth explaining because it is central to the guide. During autoregressive generation, a transformer caches the key and value tensors for tokens it has already processed, avoiding recomputation at every step. This cache grows with both context length and the number of concurrent sequences, and on a model of this size it can approach the weights themselves in memory footprint at long contexts. Managing it carefully is therefore essential to fitting the workload onto four cards while preserving usable context length.

The article's emphasis on "hybrid configurations" likely refers to strategies that combine different treatments of the cache — for example, mixing precisions, offloading portions to system memory, or accommodating models whose layers use different attention patterns. The exact details would depend on the author's setup, but the general aim is to trade some latency or added complexity for the