C-3POが自己観測と学習を統合 ── DuckDBとThompson Samplingでv1.0超え A personal AI agent project called C-3PO closed its self-observation and learning loop by …

A solo-developed AI agent called C-3PO has reached a milestone where its self-observation and learning loop is fully closed, according to a recent post by its author. The new version combines DuckDB-based action logging with Thompson Sampling bandit learning, allowing the agent to update its strategies from the outcomes of its own past decisions rather than relying solely on static prompts.

The article focuses on what changed since v1.0. The earlier version appears to have emphasized prompt design and tool invocation, while the new build introduces a persistent action log in DuckDB that downstream learners can query. Thompson Sampling, a classic Bayesian bandit method that draws arms from posterior Beta distributions, is used to balance exploration and exploitation. Plugging this into an LLM agent shifts decision-making from ad-hoc heuristics toward statistically grounded policy updates, which the author argues meaningfully outperforms the v1.0 baseline.

For context, the broader agent ecosystem has been moving from one-shot prompt engineering toward longer-horizon concerns such as memory, evaluation, and self-improvement. Frameworks like OpenAI's Assistants API, LangGraph, and CrewAI offer state and history primitives, but relatively few personal projects wire the full pipeline — observe, store, evaluate, and feed back into policy — end to end. DuckDB has become a popular choice for this kind of analytical layer because it is embedded, plays well with Pandas and Arrow, and handles columnar queries on local data with surprisingly low overhead. That makes it a natural fit for agent telemetry where you want SQL-style introspection without standing up a separate database.

Thompson Sampling itself has a long track record in recommendation and ad-serving systems, and a growing line of research applies bandit methods to prompt selection, tool routing, and model cascading inside LLM stacks. Compared with full reinforcement learning, bandits demand much simpler reward design and are far cheaper to run, which likely explains their appeal in hobby-scale projects. That said, real deployments have to grapple with how to define rewards in open-ended tasks, how to cope with non-stationary environments where the best action drifts over time, and how to avoid overfitting to noisy short-term signals. None of these are solved problems, and the post does not claim otherwise.

What makes the write-up interesting is less the novelty of any single component and more the demonstration that an individual developer can stitch the pieces together into a coherent loop. The pattern — log every decision and outcome to a local analytical store, then let a lightweight learner adjust future choices — is broadly applicable beyond this particular agent. It mirrors, in miniature, the kind of feedback infrastructure that larger AI labs build around their production systems, and suggests that similar techniques are increasingly accessible to indie builders. Readers experimenting with their own agents may find the DuckDB plus Thompson Sampling combination a practical starting point, though results will depend heavily on how rewards are framed for the specific task.