AIエージェントの「判定」問題、検査工程はとっくに解いていたThis article argues that the challenge of having AI agents reliably evaluate…

AI agents increasingly carry out multi-step tasks on their own—writing and running code, querying data, drafting and revising documents—and at each step they must decide whether a result is good enough to continue. A recent post on the Japanese developer platform Zenn argues that this "judgment" problem, in which an agent must evaluate the correctness of its own output, is far from novel. It is, the author contends, essentially the same problem that manufacturing inspection has been refining for decades, and the field's accumulated practices offer concrete patterns for designing evaluation into agent systems.

The underlying difficulty is familiar to anyone building with large language models. Asking a model to grade its own answer—commonly framed as self-evaluation or "LLM-as-judge"—is often unreliable. A model that generated a flawed answer frequently lacks the means to detect the flaw, and a model's expressed confidence does not map cleanly onto whether it is actually right. This matters because agentic workflows chain many steps together, so an error accepted early can propagate and compound, leaving the final result hard to trust and hard to debug.

The article's central point is that factories confronted the same structural issue long ago and developed durable answers. One is the separation of making from checking: on a well-run line, the process that produces a part and the process that inspects it are kept independent, precisely because a maker is poorly placed to catch its own mistakes. Applied to agents, this suggests that a separate verifier—a different model, a rule set, or a deterministic test—is more trustworthy than asking the same model that produced the work to bless