[AWS感情分析選手権2026] Comprehend・Sonnet 4.6・Opus 4.8に「微妙な日本語」を感情分析させてみた A hands-on benchmark pitting AWS Comprehend, Claude Sonnet 4.6, and Opus 4.8 against nuanc…

A recent hands-on blog post on Qiita, framed as an informal "AWS Sentiment Analysis Championship 2026," pits three different approaches against one of the harder problems in Japanese natural language processing: detecting sentiment in text that relies on sarcasm, euphemism, and understatement. The exercise matters because sentiment analysis underpins many production systems, from customer-feedback dashboards to social-media monitoring, yet much of that tooling is benchmarked on clearer, more direct text than real-world Japanese tends to be.

The three contenders represent two distinct philosophies. On one side is Amazon Comprehend, AWS's managed natural language service, which exposes a purpose-built sentiment endpoint that returns a label (Positive, Negative, Neutral, or Mixed) along with confidence scores for each category. On the other side are two large language models from Anthropic's Claude family, referred to in the post as Sonnet 4.6 and Opus 4.8, which are prompted to classify sentiment rather than calling a dedicated API. This pairing is a useful comparison because it captures a choice many teams now face: rely on a specialized, lower-cost classifier, or hand the task to a general-purpose model that reasons over context.

The difficulty the author targets is specific to high-context languages. Japanese frequently conveys negative meaning through indirect phrasing, where a literal reading points one way and the intended sentiment points the other. Stock expressions such as a polite "検討します" ("we will consider it"), which often functions as a soft refusal, or exaggerated praise delivered sarcastically, are notoriously hard for systems that map surface words to sentiment. Understatement compounds the problem, since a mildly worded complaint may carry strong dissatisfaction. These are exactly the cases where a model's ability to infer speaker intent, rather than tally positive and negative tokens, becomes decisive.

According to the post, the results expose a clear split in strengths and weaknesses rather than a single dominant winner. Comprehend appears to perform well on straightforward, explicitly worded sentences and offers advantages in speed, predictable cost, and operational simplicity as a fully managed service. However, it is described as more prone to taking sarcastic or euphemistic statements at face value, classifying ironic praise as positive or missing the negative subtext in polite deflections. The Claude models, by contrast, are reported to handle these subtle cases more often, likely because their broader contextual reasoning lets them weigh tone and pragmatics. The trade-off noted is that an LLM can sometimes overinterpret neutral or factual statements, reading emotion into text that carries none, and it brings higher latency and per-call cost.

The reported gap between the two Claude versions is more incremental, with the larger Opus model generally edging ahead on the trickiest items while the smaller Sonnet model offers a closer balance of accuracy and efficiency. Readers should treat these findings as indicative rather than definitive. A single-author benchmark on a curated set of difficult sentences is not a controlled study; outcomes can shift substantially with prompt wording, the choice of examples, the sentiment label scheme, and how borderline or "mixed" cases are scored. The post is best read as an illustrative probe of behavior, not a ranking that generalizes to all workloads.

For context, this comparison sits within a broader industry pattern of LLMs encroaching on tasks once served by dedicated NLP services. Comprehend competes with similar managed offerings such as Google Cloud's Natural Language API and Azure AI Language, all of which provide classifier-based sentiment scoring. Anthropic's models, alongside OpenAI's and Google's, increasingly absorb these classification jobs through prompting, few-shot examples, or structured output. The practical decision often comes down to volume, budget, and tolerance for error: high-throughput pipelines may favor a cheap classifier with occasional misses, while applications where misreading tone is costly may justify an LLM, or a hybrid that routes ambiguous cases to a stronger model.

The takeaway from the experiment is less about declaring a champion and more about matching the tool to the text. For clean, direct Japanese, a managed service may be sufficient and economical. For content saturated with irony and indirect expression, a reasoning-capable model appears better suited, provided teams account for its cost, latency, and tendency to over-read neutral statements. Anyone adapting these findings would do well to build their own labeled evaluation set drawn from their actual domain before committing to an architecture.