コンパニオンAIの記憶を、普通のRAGじゃない設計にした話 A developer shares an experimental memory architecture for a companion AI that departs fro…

Designing memory for a companion AI, an agent meant for casual conversation and emotional presence, is more than a retrieval-accuracy problem. This article shares one developer's experiment in building a memory system that deliberately avoids the standard RAG pattern, arguing that a generic vector store does not capture what makes a companion feel continuous and personal.

A typical RAG pipeline embeds the user's utterance, retrieves the most similar chunks from a vector database, and injects them into the prompt. That works well for FAQ-style or knowledge-grounded assistants, but the author suggests it struggles for companion use cases. When a user says something like "let's continue yesterday's conversation," pure semantic similarity has a hard time reconstructing temporal flow or topic threads, because nearest-neighbor search has no native notion of episode or relationship.

Instead of cramming everything into one embedding index, the post points toward a structured approach: separating memories by type, such as episodic events, user preferences, and evolving relational state. The intent, as far as can be inferred, is to move from "retrieval" toward something closer to "recollection," where the agent surfaces memories based on contextual cues rather than raw cosine similarity.

This line of thinking is not isolated. Projects like MemGPT and Letta have explored hierarchical memory with explicit working and long-term tiers, while LangChain and LlamaIndex expose memory abstractions that combine summarization, entity tracking, and vector recall. Commercial companion services such as Character.AI and Replika are widely believed to use similar layered architectures, often pairing compressed summaries with structured user profiles and emotion tags. The design discussed here appears to sit within that broader tradition.

For local LLM deployments, structured memory has a practical payoff beyond fidelity. By keeping persistent state outside the prompt and only injecting what is relevant, developers can maintain a coherent persona even with smaller context windows. That matters when running 7B or 13B class models on consumer hardware, where every token of context competes with reasoning headroom. A well-designed memory layer effectively extends the model's apparent capacity without forcing a larger base model.

That said, structured memory introduces its own difficulties. Deciding when to write, when to update, and how to reconcile contradictions, for example when a user changes their stated preferences, is non-trivial. There are also privacy considerations: a companion that remembers intimate details indefinitely raises questions about retention policies, export, and deletion that the broader industry is still working through. Hallucinated memories, where the model confidently recalls events that never happened, remain a known failure mode that purely vector-based systems tend to amplify.

Companion AI design is still a young field without settled best practices, and much of the progress so far has come from individual developers sharing implementation notes rather than from formal benchmarks. Posts like this one contribute to that emerging body of practical knowledge, and the proposed direction, treating memory as a structured, multi-layered subsystem rather than a single retrieval call, is consistent with where the wider ecosystem appears to be heading.