「同じ内容・構成違いのExcel」を5つの生成AIに照合させてみたAn experiment comparing five generative AIs on the task of cross-referencing…

Cross-referencing spreadsheets that contain the same underlying data but arrange it differently is one of the most common and error-prone tasks in office work, and it is exactly the kind of problem that large language models are increasingly being asked to handle. A recent experiment published on Qiita set out to test how well five generative AI systems perform this job, feeding each of them Excel files that hold identical information organized under different layouts and asking them to reconcile the discrepancies. The results, the author reports, reveal meaningful differences in accuracy across the models, which matters for anyone considering where AI can safely fit into spreadsheet-heavy workflows.

The core challenge is deceptively simple. Two files might record the same sales figures, customer records, or inventory counts, but one could place totals in columns while another uses rows, apply different header labels, merge cells, or reorder entries. A human reviewer intuitively maps these structures onto one another, but a machine has to infer that "Q1 Revenue" in one sheet corresponds to "Sales, Jan–Mar" in another, then confirm that the values actually match. This tests several capabilities at once: reading tabular data reliably, understanding semantic equivalence between differently named fields, and performing exact numerical comparison without hallucinating or silently dropping rows.

According to the write-up, the five models did not perform equally, and the gaps were significant enough to affect practical trust. Some systems appear to handle layout normalization well but stumble on precise value matching, while others parse the raw cells accurately yet struggle when the structures diverge sharply. This pattern is consistent with a broader known limitation of language models: they are strong at pattern recognition and natural-language reasoning but can be unreliable at deterministic, arithmetic-style verification, where a single transposed digit constitutes a hard failure rather than a close-enough answer.

A key technical distinction that shapes these outcomes is whether a model reasons over the spreadsheet as flattened text or executes actual code to process it. Several current AI platforms, including tools built around Claude and other frontier models, now offer code-execution or analysis environments that can load a file with libraries such as pandas and compute comparisons programmatically. When a model writes and runs code to align and diff two tables, its accuracy on exact matching tends to be far higher than when it merely "reads" the numbers and answers from inference. Readers evaluating the experiment should therefore consider not only which model was used but also whether it had tool access, since that often matters more than the underlying model's raw reasoning ability.

The context here connects to a wider industry push toward AI-assisted data work. Microsoft has been integrating Copilot directly into Excel, OpenAI and Anthropic both offer file-upload and analysis features, and a growing ecosystem of add-ins promises automated reconciliation, cleanup, and reporting. For finance, auditing, and operations teams, reliable cross-referencing is not a novelty but a daily necessity, and the cost of a missed mismatch can be substantial. That raises the stakes for the kind of benchmarking this article attempts, because vendor demonstrations rarely stress-test messy, real-world layout variation.

There are caveats worth keeping in mind. An informal comparison of five models on a set of files is a useful signal rather than a rigorous benchmark; results can shift with prompt wording, file complexity, model versions, and whether the data fits within a model's context window. Larger or more intricate spreadsheets may expose weaknesses that small examples hide, and models are updated frequently enough that any ranking is likely to be a snapshot rather than a durable verdict. The experiment's value lies more in illustrating the failure modes and the variability than in crowning a definitive winner.

For practitioners, the practical takeaway is to treat generative AI as an accelerator for spreadsheet reconciliation rather than an unsupervised authority. Using models that can execute code, keeping source files clean and well-labeled, and spot-checking results against known totals all reduce risk. Experiments like this one are helpful precisely because they encourage a clearer, evidence-based sense of what these tools can and cannot yet be trusted to do.