ローカルLLM(Ollama)にJSONを厳密に返させる — 自分専用ニュースbot開発記 #1This article explains how to enforce strict JSON output from a local LLM…

Running large language models locally has become increasingly practical, and Ollama is one of the most popular tools for doing so. Yet anyone who tries to build an automated pipeline on top of a local model quickly runs into a familiar obstacle: coaxing the model into returning clean, machine-readable JSON that a downstream program can parse without breaking. This article, the first in a series documenting the construction of a personal news bot, describes how to enforce strict JSON output using Ollama's structured output feature rather than relying on prompt wording alone.

The core issue is that instruction-tuned models are trained to be conversational. Even when a prompt explicitly asks for JSON, a model may wrap the response in Markdown code fences, prepend a sentence such as "Here is the JSON you requested," omit a required field, or emit subtly invalid syntax like trailing commas or unescaped quotes. For a one-off interaction a human can clean this up by hand, but for an unattended bot that runs on a schedule, a single malformed response can halt the entire pipeline. Prompt engineering lowers the failure rate but rarely eliminates it, and the failures tend to appear unpredictably, which makes them hard to test against.

Structured output addresses this at the decoding level rather than the prompting level. Ollama exposes a format parameter in its API that accepts a JSON Schema object describing the exact shape of the expected response: the field names, their data types, and which fields are required. When this schema is supplied, the runtime constrains token generation so that only tokens consistent with the schema can be produced. In practice this is implemented through constrained or grammar-based sampling, a technique that llama.cpp, the inference engine underpinning Ollama, supports through GBNF grammars. Because invalid tokens are masked out during generation, the model is prevented from producing output that violates the declared structure. That is a considerably stronger guarantee than asking the model politely and hoping it complies.

In a typical workflow, a developer defines the schema, often generated automatically from a Pydantic model in Python or a Zod schema in TypeScript, and passes it in the request alongside the prompt. The response then arrives as a string that reliably parses into the expected object. It is worth noting that constraining the format does not guarantee the content is correct; the model can still populate a valid field with an inaccurate value or a hallucinated summary. Structured output solves the parsing problem, not the reasoning problem, so validation of the actual values still matters. It also appears that smaller local models benefit especially from this approach, since they are more prone to formatting drift than larger hosted models.

This capability sits within a broader industry movement toward reliable structured generation. OpenAI introduced its Structured Outputs feature in 2024, offering a similar schema-adherence guarantee for its hosted models, and earlier JSON mode options provided a weaker version of the same idea. In the open-source ecosystem, libraries such as Outlines, Guidance, and Instructor tackle the problem from different angles, ranging from grammar enforcement to automatic retries and validation against a schema. Ollama's built-in support means that developers no longer need to bolt on an external library for basic cases, which lowers the barrier for local-first projects that prioritize privacy or offline operation.

For the news bot use case, this reliability is the foundation everything else depends on. The bot is likely to fetch articles, ask the model to extract fields such as a title, a concise summary, a category, and a sentiment label, and then store or route those results. Any of those steps that assumes well-formed JSON would be fragile without a hard structural guarantee. By moving the constraint into the model's decoding process, the pipeline can treat the LLM more like a predictable component and less like an unpredictable text generator.

Readers considering a similar build should keep a few prerequisites in mind. A working knowledge of JSON Schema helps, since overly strict or ambiguous schemas can occasionally reduce output quality or cause the model to stall. Choosing a model that handles structured generation well is also worth testing, as behavior varies across model families and sizes. Subsequent entries in the series are expected to cover the surrounding components, such as source ingestion, scheduling, and delivery, building on this structured-output groundwork.