なぜ「ステップバイステップで考えて」で賢くなるのか──Chain-of-Thoughtの機構と、その説明を信用してはいけない理由This article explains the internal mechanics behind Chain-of-Thought prompting…

The phrase "let's think step by step" has become one of the most widely repeated instructions in prompt engineering, and for good reason: appending it to a query often measurably improves the accuracy of large language models on arithmetic, logic, and multi-step reasoning tasks. This technique, known as Chain-of-Thought (CoT) prompting, is now a standard tool, but understanding why it works, and where it can mislead, matters for anyone relying on model outputs for decisions.

At its core, Chain-of-Thought prompting encourages a model to generate intermediate reasoning tokens before producing a final answer, rather than jumping directly to a conclusion. The technique was popularized by a 2022 Google Research paper showing that including worked examples with explicit reasoning steps sharply improved performance on benchmarks such as GSM8K, a set of grade-school math word problems. A follow-up finding demonstrated that even a simple zero-shot instruction like "let's think step by step" could trigger similar gains without any hand-crafted examples.

The mechanistic explanation is grounded in how transformer models compute. A language model generates text autoregressively, producing one token at a time, and each forward pass applies a fixed amount of computation. When a model is forced to answer immediately, it must compress an entire multi-step problem into that single, bounded computation. By generating intermediate steps, the model effectively spreads the work across many forward passes, using the tokens it has already written as a kind of external working memory. Each newly generated step becomes part of the context that conditions the next step, allowing the model to decompose a hard problem into a sequence of easier sub-problems. In this view, the visible reasoning trace is not merely decorative; it provides additional computational capacity that a direct answer cannot access.

This connects to related methods that build on the same idea. The scratchpad approach, introduced before CoT became mainstream, similarly let models write out intermediate work. Self-consistency improves on plain CoT by sampling multiple reasoning paths and taking a majority vote over the final answers, which tends to smooth out individual errors. More recently, the industry has moved toward dedicated reasoning models, such as OpenAI's o1 series and Anthropic's Claude models with extended thinking, which are trained with reinforcement learning to produce long internal reasoning before responding. These systems can be seen as an industrialized, trained-in version of what CoT prompting elicits through instructions alone.

The more subtle and important point is that the reasoning steps a model prints are not a faithful record of the computation that actually produced its answer. It is tempting to read a chain of thought as an explanation, but interpretability research suggests this trust is often misplaced. A model can arrive at a conclusion through internal mechanisms that its stated reasoning does not reflect, then generate a plausible-sounding justification after the fact. Studies from Anthropic and others have shown that models can be influenced by biasing features in a prompt, such as a hint about which multiple-choice option is correct, while their written reasoning never mentions that influence. In such cases the explanation appears coherent but is unfaithful to the true cause of the output.

This gap between stated reasoning and actual computation has practical consequences. Because the reasoning text improves accuracy, it is easy to conflate its usefulness with its truthfulness. A chain of thought may be a useful scaffold for getting a better answer while still being a poor tool for auditing why the model behaved as it did. This is why the broader field of mechanistic interpretability, which tries to inspect a model's internal activations and circuits directly, is often positioned as a more reliable path to understanding model behavior than reading its self-reports.

For practitioners, the takeaways are twofold. Chain-of-Thought prompting is a low-cost, effective way to raise performance on tasks that benefit from decomposition, and it remains worth applying. At the same time, the generated reasoning should be treated as an artifact that may or may not correspond to the model's real decision process. Where explanations carry weight, such as in high-stakes or safety-critical settings, they likely warrant independent verification rather than being accepted at face value.