ollama は長い入力の中間部分を無音で切り捨てる — 実測で半分しか処理されない問題Benchmarking reveals that ollama silently drops the middle portion of inputs…

Running large language models locally has become far easier thanks to tools like Ollama, but a recently reported benchmarking result highlights a subtle behavior that can quietly undermine results: when an input exceeds the default context length, Ollama appears to discard the middle portion of the prompt without emitting any warning. According to the testing described, roughly half of the supplied content can be dropped, leaving the model to reason over an incomplete version of what the user believes it received. For anyone feeding long documents, transcripts, or codebases into a local model, this is a meaningful source of silent error.

The root of the issue is the context window, the maximum number of tokens a model can consider at once. In Ollama this is governed by the num_ctx parameter, and the default has historically been set to a conservative value, commonly 2048 tokens, regardless of the much larger context lengths that modern models advertise. A model such as Llama 3 may support tens of thousands of tokens, but if Ollama is invoked with its default configuration, the effective window can remain small. When the prompt is longer than that window, the runtime must decide what to keep.

What the benchmarking reportedly uncovered is that the truncation is not a simple cut from the end. Instead, the beginning and the end of the input are retained while the middle is removed. This strategy is not arbitrary. Keeping the opening preserves any system prompt or task instructions, and keeping the tail preserves the most recent turns of a conversation, which are often the most relevant. The trade-off is that information buried in the center of a long input, such as a clause in the middle of a contract or a function definition halfway through a file, can vanish entirely. Because no error or notice is produced, the user has no obvious signal that the model never saw the omitted text.

This behavior is easy to miss precisely because the model still produces a fluent, confident answer. It simply answers based on the portion it retained. In retrieval-augmented or long-context workflows, that can translate into subtly wrong summaries, missed instructions, or hallucinated details that the user attributes to model weakness rather than input truncation. The practical takeaway from the report is that explicitly setting num_ctx to match the intended input size is essential, whether through the API request options, a Modelfile parameter, or an environment configuration. Raising the value does increase memory consumption, since the key-value cache grows with context length, so users must balance capacity against available RAM or VRAM.

Some context helps explain why this happens. Ollama is built on top of llama.cpp, the widely used C++ inference engine, and it inherits both its performance characteristics and its context-management conventions. llama.cpp and similar runtimes implement context-handling strategies to cope with inputs that outgrow the allocated window, and truncation from the middle is one such approach. The behavior is arguably a reasonable default for chat, where recency matters, but it becomes a liability for document processing, where every part of the input may carry weight. Comparable local-inference frameworks, including vLLM, LM Studio, and text-generation-inference, each handle overflow differently, and some will instead raise an explicit error when a request exceeds the configured limit. That difference in philosophy, failing loudly versus degrading silently, is at the heart of the concern.

For users, a few defensive practices reduce the risk. Measuring token counts before sending a request, setting num_ctx deliberately for the task at hand, and verifying that the loaded model actually supports the requested length all help. It is also worth confirming behavior after upgrades, since defaults and truncation logic can change between releases, and what is described here may not hold identically across every version or model. The broader lesson extends beyond a single tool: local LLM stacks involve several layers, from the model file to the runtime to the client, and each can impose limits that are not surfaced clearly. Treating the context window as a hard constraint to be managed explicitly, rather than an invisible detail handled automatically, is the most reliable way to ensure a model receives the full input it is expected to process.