RAGとローカルLLMとは何か？AIエージェントに「外部の知識」をどう持たせるかThis article concludes a series on AI agent design by explaining RAG and local…

Retrieval-Augmented Generation (RAG) and locally hosted large language models are two of the most practical techniques for giving AI agents access to knowledge that lies outside their training data. This article closes a side series on AI agent design that has previously examined Harness, Loop Engineering, Context Engineering, and Memory, turning finally to how an agent can reach beyond what its underlying model already knows. The topic matters because even the strongest general-purpose models have a fixed knowledge cutoff and no awareness of private, proprietary, or very recent information, which limits how far they can be trusted in real workflows.

At its core, RAG addresses a simple problem: a language model can only reason over what it was trained on plus whatever text sits in its current context window. When a question depends on a company's internal documents, a product manual, or data published after training, the model may guess or fabricate an answer. RAG mitigates this by retrieving relevant material from an external source at query time and inserting it into the prompt, so the model generates its response grounded in supplied evidence rather than memory alone. In practice this tends to reduce hallucination and makes answers easier to verify, because the retrieved passages can be cited.

The retrieval step usually relies on vector search. Documents are split into chunks, and each chunk is converted into an embedding, a numerical vector that captures its semantic meaning. Those vectors are stored in a vector database or index. When a user asks a question, the query is embedded the same way, and the system finds the chunks whose vectors are closest to it, typically by cosine similarity or a related distance measure. Those top matches become the context handed to the model. This design is what allows a knowledge base to be searched by meaning rather than by exact keyword matching, and it is why the vector-search and knowledge-base concepts appear alongside RAG so frequently.

Building a functional RAG pipeline involves several moving parts that each affect quality. Chunk size and overlap influence whether retrieved passages contain enough context. The choice of embedding model affects how well semantic similarity is captured. Retrieval strategies such as hybrid search, which blends keyword and vector methods, or re-ranking, which reorders candidate passages before they reach the model, are common refinements. Popular frameworks including LangChain and LlamaIndex, along with vector stores such as FAISS, Chroma, Pinecone, and pgvector, provide building blocks for these stages, though the general pattern is independent of any single tool.

Local LLMs form the second half of the discussion. Rather than calling a hosted API, a local model runs on your own hardware or infrastructure. Open-weight families such as Llama, Mistral, Qwen, and Gemma, served through runtimes like Ollama, llama.cpp, or vLLM, have made this increasingly accessible. The motivations are typically data privacy, since sensitive documents never leave the environment, along with cost predictability, offline capability, and control over model versions and behavior. The tradeoffs are also real: local models often lag the largest hosted systems in raw capability, and running them well requires attention to hardware, quantization, and throughput.

The two techniques complement each other. Pairing RAG with a local model lets an organization keep both its private knowledge base and the reasoning process inside its own boundaries, which is appealing for regulated or security-conscious settings. For an AI agent, this combination effectively supplies a working memory of external facts that can be updated simply by changing the underlying documents, without retraining the model itself. That flexibility is why RAG is frequently described as one of the most cost-effective ways to specialize a general model for a specific domain.

Placed within the broader series, RAG and local LLMs can be read as the knowledge layer that sits beneath the earlier themes. Memory concerns what an agent retains across interactions, Context Engineering concerns how information is arranged in the prompt, and RAG concerns where trustworthy external information comes from in the first place. Taken together, these pieces sketch a fuller picture of how modern agents are assembled. Readers should note that the field is moving quickly, so specific tools and benchmarks are likely to shift, even as the underlying retrieval-and-generation pattern appears to remain durable.