Claude Fable 5 を9Bモデルに蒸留？ 100万トークンの超長文推理モデル「Qwythos-9B」を4GBのVRAMで動かすQwythos-9B is a purported Claude Fable 5 distillation that supports 1M-token…

Qwythos-9B is a recently publicized local language model that, according to a Qiita blog post, was distilled from Anthropic's "Claude Fable 5" and is said to handle context windows of up to one million tokens while running on consumer GPUs with as little as 4 GB of video memory. The appeal is obvious: if the description holds up, it would place extremely long-context reasoning within reach of ordinary laptops and budget desktop cards, rather than requiring cloud APIs or datacenter-class accelerators. At the same time, the specifics are unverified, and a few of them sit in tension with what is publicly known about the underlying techniques.

Start with the distillation claim. Model distillation is a well-established method in which a smaller "student" model is trained to reproduce the outputs, and sometimes the internal probability distributions, of a larger "teacher." Done well, it can transfer much of a big model's behavior into a far cheaper package. The stated teacher here, "Claude Fable 5," does not correspond to any publicly documented Anthropic release; the company's current lineup uses the Opus, Sonnet, and Haiku names. Distilling from a proprietary, API-only model also typically means learning from its generated text rather than its weights, an approach that raises both quality questions and potential terms-of-service concerns. Readers should treat the provenance as a claim rather than a confirmed fact. The "Qw" prefix in the name may hint at a lineage related to Alibaba's Qwen family, though that too is unconfirmed.

The one-million-token context figure is the headline feature, and it is where technical scrutiny should focus. Reaching very long contexts generally relies on techniques such as rotary position embedding scaling, sparse or sliding-window attention, and streaming approaches that avoid recomputing the full sequence. The harder problem is memory. A transformer must store a key-value (KV) cache for every token it attends to, and for a 9-billion-parameter model that cache can grow into the tens of gigabytes at a million tokens, well beyond a 4 GB budget. Fitting both the model weights and a usable context into 4 GB would require aggressive quantization of the weights (likely 4-bit or lower), heavy KV-cache compression or quantization, and probably offloading part of the workload to system RAM or disk. Any of these can work, but they usually trade throughput or accuracy for the memory savings, so real-world performance at the full context length is worth verifying independently.

For background, the broader field has been moving quickly on both fronts. On context length, Google's Gemini 1.5 models advertised windows of a million tokens and beyond, while Anthropic's Claude and OpenAI's GPT-4-class models have offered windows in the hundreds of thousands. On the local side, quantization formats such as GGUF, used by llama.cpp, along with GPTQ and AWQ, have made it routine to run 7B-to-9B models on modest hardware. Compact, capable open models like Qwen2.5, Llama 3, Mistral, and Microsoft's Phi series have pushed what is possible on a single consumer GPU. Qwythos-9B, as described, appears to sit at the intersection of these trends rather than representing an isolated breakthrough.

None of this makes the model implausible; small models with long, if sometimes shallow, context handling are an active research area, and distillation plus quantization is a sensible recipe for the goal. But claims of a million-token window on 4 GB of VRAM deserve concrete benchmarks: retrieval accuracy across the full context (for example, "needle in a haystack" tests), tokens-per-second at various context lengths, and the exact quantization and offloading configuration used. Until such measurements and the model itself are available for public testing, Qwythos-9B is best understood as an intriguing but unconfirmed report. If it delivers even a fraction of the stated capability on the stated hardware, it would be a notable data point in the ongoing eff