Tencent WorkBuddy Bench: 汚染耐性タスク構築を備えたマルチドメインコーディングエージェントベンチマークTencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction

Tencent has published WorkBuddy Bench, a benchmark for evaluating large language model coding agents across several software domains, built around a task-construction method intended to resist data contamination. As coding agents shift from research demonstrations toward tools that developers use for real engineering work, the trustworthiness of the benchmarks used to compare them has become a pressing concern. WorkBuddy Bench is positioned as a response to that concern, aiming for evaluation results that are harder to inflate and easier to interpret.

The central issue the work addresses is contamination, which occurs when the problems and solutions used to test a model have already appeared, in some form, in its training data. When that happens, a system may appear to reason through a task while actually reproducing memorized answers, and reported scores overstate genuine capability. This problem has grown more acute as models are trained on enormous web-scale corpora that include public code repositories, issue trackers, and the benchmark datasets themselves. For coding tasks in particular, where popular test sets are hosted openly and widely discussed, the risk that a benchmark has leaked into training is substantial.

According to Tencent's summary, WorkBuddy Bench tackles this with a contamination-resistant task construction approach combined with coverage of multiple domains. Multi-domain design matters because narrow benchmarks can reward agents that are tuned to a single style of problem, such as algorithmic puzzles, while saying little about performance on the varied work a professional engineer encounters. Spanning several domains is intended to produce a more representative picture of an agent's abilities across different codebases, languages, and task types. The contamination-resistant element appears aimed at ensuring that tasks are new or transformed enough that memorization provides little advantage, though the specific construction pipeline is detailed in the paper itself rather than the summary.

Techniques that benchmarks in this space typically use to limit contamination include drawing on freshly created or recently updated material, generating novel problem variants programmatically, and continuously refreshing the task pool so that any given snapshot is unlikely to overlap with a model's training cutoff. Approaches like these are what allow a benchmark to claim greater resistance to leakage, and WorkBuddy Bench is likely to rely on comparable strategies, given its stated goals. The broader intent is to make results fairer when comparing agents built on different underlying models, since a contaminated benchmark can favor whichever system happened to ingest the test data.

The work sits within a crowded and fast-moving area of evaluation research. HumanEval and MBPP established early standards for function-level code generation, but their small size and public availability made them prime candidates for contamination. SWE-bench shifted attention toward realistic, repository-scale tasks derived from actual GitHub issues, and its variants have become a common reference point for agentic coding systems that must navigate a codebase, edit multiple files, and run tests. LiveCodeBench and similar efforts emphasize using problems released after a model's training date as a practical guard against leakage. WorkBuddy Bench's combination of multi-domain breadth and an explicit contamination-resistant methodology places it in dialogue with these prior benchmarks rather than replacing any single one.

The release also reflects broader industry momentum around coding agents, which differ from simple code-completion tools by planning, executing commands, and iterating toward a goal with limited human intervention. Major labs and companies have invested heavily in such systems, and Tencent's contribution of an evaluation benchmark, rather than only a model, signals interest in the infrastructure that supports credible comparison. Reliable measurement is a prerequisite for progress, because teams optimize toward whatever their benchmarks reward.

Several caveats are worth noting. No benchmark fully eliminates contamination risk, and the effectiveness of any contamination-resistant design depends on execution details, the freshness of its tasks over time, and how faithfully its problems mirror real engineering demands. Independent adoption and replication by outside researchers will be important indicators of how well WorkBuddy Bench holds up in practice. As with any benchmark introduced by an organization that also develops competing models, external validation will help establish whether the evaluation is as neutral and durable as intended. The paper's full methodology, task statistics, and reported results provide the basis for such scrutiny.