社内データ分析エージェント「Qubot」の構築事例How we built an internal data analytics agent

GitHub has published a detailed account of how it built Qubot, an internal Copilot-powered analytics agent that lets any employee ask questions about company data in plain language rather than writing queries by hand. The case study matters because it offers a concrete look at how a large engineering organization is moving natural-language data access from demo to daily production use, and what trade-offs that journey involves.

At its core, Qubot is designed to translate everyday questions into structured queries against GitHub's internal data, returning answers, tables, or visualizations without requiring the asker to know SQL or the underlying schema. According to the post, the goal is to broaden access to data so that product managers, support staff, and others outside the data team can self-serve, reducing the bottleneck of routing every analytical request through a small group of analysts. This is a familiar problem in many companies, where data exists but the expertise to retrieve it is concentrated.

The technical pattern Qubot appears to follow is what the industry broadly calls a text-to-SQL or analytics agent. Such systems typically combine a large language model with contextual information about the data warehouse, including table names, column definitions, relationships, and business-specific terminology. A key lesson GitHub emphasizes is that raw schema access is rarely enough. Models need a curated semantic layer or metadata catalog that encodes how the business actually defines metrics, because terms like active user or revenue can carry meanings that are not obvious from column names alone. Without that grounding, an agent can produce queries that run successfully but return subtly wrong answers, which is more dangerous than an outright failure.

The post also reflects on reliability and trust, recurring themes for any agent that touches business data. Because language models can hallucinate or misinterpret ambiguous questions, the team appears to have invested in mechanisms that show their work, such as surfacing the generated query, citing which tables were used, and allowing users to inspect and correct the logic. Letting people verify the path to an answer is presented as essential for adoption, since users are unlikely to act on numbers they cannot trust. Evaluation is described as an ongoing discipline rather than a one-time benchmark, with the team building test sets of representative questions to measure accuracy as the system and the data evolve.

Several practical engineering takeaways emerge. Scoping the agent to well-understood, high-value datasets first is likely more productive than trying to cover every table at once. Handling ambiguity gracefully, by asking clarifying questions instead of guessing, improves both accuracy and user confidence. Performance and cost also matter, since each natural-language query may trigger multiple model calls, and caching or query optimization helps keep the experience responsive. Governance is another thread: an internal tool that can read company data must respect existing access controls so that users only see what they are permitted to see.

Qubot fits into a wider movement around agentic and conversational analytics. Adjacent efforts include offerings and patterns from data platform vendors and a growing set of open frameworks for building LLM agents, along with emerging standards such as the Model Context Protocol that aim to give models structured, governed access to external tools and data sources. The broader Copilot ecosystem, which GitHub has extended from code completion toward chat, agents, and workspace integrations, provides the foundation the team appears to have built on, reusing components for prompting, tool calling, and orchestration.

For readers considering similar projects, the underlying prerequisites are worth noting: a reasonably clean data warehouse, a maintained semantic or metrics layer, clear access policies, and a feedback loop for catching errors. The honest framing of the GitHub post, focusing as much on limitations and lessons as on capabilities, is a useful signal that these systems require sustained investment rather than a single launch. As natural-language interfaces to data continue to mature across the industry, case studies like Qubot help clarify where the real engineering challenges lie, and how organizations can balance accessibility with the accuracy and trust that analytical work demands.