nanochatで理解するLLM製造工程A hands-on technical book that walks through every stage of LLM…

A new Japanese-language technical book sets out to demystify how large language models are actually made, using Andrej Karpathy's nanochat codebase as its guide. For readers who have only ever interacted with LLMs through an API or a chat window, the premise is notable: rather than treating model training as an inaccessible activity confined to well-funded labs, it argues that the entire pipeline can now be read, understood, and even run on a personal machine.

The book is built around nanochat, a compact codebase of roughly 8,000 lines released by Karpathy. Karpathy, a well-known figure in the field who previously led AI at Tesla and was a founding member of OpenAI, has a track record of producing minimal, educational implementations. Earlier projects such as minGPT and nanoGPT distilled transformer training into readable code, and his "Zero to Hero" lecture series walked through building neural networks from scratch. nanochat continues that lineage as a full-stack example that spans the whole path from raw text to a working chat assistant.

Across eight chapters, the book follows that pipeline stage by stage in code: tokenizer training, pretraining, supervised fine-tuning (SFT), reinforcement learning, evaluation, and the inference engine that ultimately serves responses. This ordering mirrors how modern chat models are constructed in practice. Tokenizer training establishes how text is broken into units; pretraining teaches the base model general language patterns; SFT adapts it to follow instructions; reinforcement learning further shapes its behavior; and evaluation and inference cover measurement and deployment. By reading the actual implementation rather than a high-level description, the book aims to make each of these steps concrete.

One of the book's framing points concerns cost. It highlights that training a GPT-2-class model, which reportedly cost around $43,000 in 2019, has become dramatically cheaper as hardware and methods have improved. The exact figure a reader will pay today depends on the configuration and compute they choose, but the broader trend the book points to is that reproducing capabilities once considered expensive is now within reach of individuals and small teams. This shift in economics is central to the argument that the pipeline is no longer the exclusive domain of large organizations.

The book also emphasizes accessibility at the low end. It walks readers through a minimal experience that can run even on a MacBook via a script called runcpu.sh, letting people exercise the pipeline without specialized accelerators. This is a scaled-down path rather than a way to train a competitive model, but it lowers the barrier to seeing the machinery in motion. The overarching goal, as the book frames it, is to help readers move from being users of LLMs to owning the process behind them.

That framing places the book within a broader movement toward local and open LLM tooling. Projects such as llama.cpp, Ollama, and various quantization techniques have made running models locally increasingly practical, and Karpathy's own llm.c explored training in pure C and CUDA. Where many of those efforts focus on inference—running an already-trained model efficiently—this book leans toward the training side, giving readers visibility into how a model comes to exist rather than only how it is served. The two perspectives are complementary, and understanding both is useful for anyone building on top of open models.

For practitioners, the value likely lies less in producing a state-of-the-art system and more in building intuition. Reading a complete, small codebase end to end can clarify how components interact—why tokenizer choices matter downstream, how fine-tuning differs from pretraining, or what an inference engine actually does. As always with fast-moving tooling, specific figures and code details may change over time, so readers should treat the numbers as illustrative of a trend rather than fixed. Still, as an educational resource, the book appears designed to turn an abstract pipeline into something readers can trace line by line and run themselves, which is arguably its most practical contribution for anyone looking to deepen their grasp of how these systems are built.