DeepSeek-V4がKVキャッシュを10分の1に削減できたCSAとHCAの設計DeepSeek-V4 addresses the memory bottleneck of KV caches in long-context LLMs…

DeepSeek-V4 targets one of the most stubborn constraints in modern large language models: the memory consumed by the key-value (KV) cache when a model handles very long inputs. A technical write-up describing the DeepSeek-V4 architecture reports that two designs, referred to as CSA and HCA, cut KV cache size to roughly one-tenth of conventional methods, a reduction the authors link to significantly improved throughput. For anyone running long-context inference on constrained hardware, the claim matters because it reframes the bottleneck from raw computation toward memory.

The motivation is grounded in how long-context serving actually behaves. The cost that bites hardest is frequently not FLOPs but the memory required to maintain the KV cache. Each token a transformer processes produces a key and a value that must be kept so later tokens can attend back to it. When an input approaches a million tokens, merely holding those per-token keys and values can fill GPU memory. Once memory is exhausted, the server must reduce batch size, and smaller batches mean fewer requests handled in parallel, which in turn lowers overall throughput. Memory pressure, rather than arithmetic, tends to set the practical ceiling.

To understand why this is hard, it helps to recall what the KV cache does. During autoregressive generation, recomputing attention over the entire history for every new token would be wasteful, so implementations cache the keys and values from previous steps. That trades compute for memory, and the memory grows with sequence length, the number of layers, and the number of attention heads. This is the pressure point that a series of prior techniques has tried to relieve.

Earlier efforts largely attacked the head dimension. Multi-Query Attention (MQA) shares a single key-value projection across all query heads, and Grouped-Query Attention (GQA), used in models such as Llama 2 and many successors, strikes a middle ground by sharing KV across groups of heads. DeepSeek's own earlier work introduced Multi-head Latent Attention (MLA), which compresses keys and values into a lower-dimensional latent representation to shrink the cache while preserving quality. The CSA and HCA designs in DeepSeek-V4 appear to continue this lineage, pushing cache compression further rather than reworking attention from scratch, though the exact mechanisms are best understood from the original description.

Based on the summary, CSA and HCA are presented as complementary architectural choices rather than a single trick, and together they are credited with the tenfold reduction. The write-up frames the benefit primarily in throughput terms, which is consistent with the memory-to-batch-size relationship: freeing cache memory allows larger batches or longer contexts on the same GPU. As with any architectural claim, independent benchmarking across tasks and context lengths would help confirm how the reduction affects accuracy, latency, and stability, and readers should treat the figures as the authors' reported results.

The broader context is an industry-wide race to serve longer contexts affordably. Systems-level tools like vLLM popularized PagedAttention, which manages KV cache in non-contiguous blocks to cut fragmentation and waste. Techniques such as sliding-window attention, quantized KV caches, and cache eviction or offloading strategies each chip away at the same problem from different angles. Model-level compression, of the kind DeepSeek describes, is attractive because it reduces the cache at its source, which can compound with serving-layer optimizations rather than competing with them.

DeepSeek has positioned itself as a prominent open-model developer, and prior releases have been notable for pairing competitive quality with efficiency-oriented engineering. If the reported one-tenth reduction holds in practice, it would streng