AIに人間用GUIを操作させるな！～正しいMCP活用法について～This article argues that having AI agents operate human-oriented GUIs is an…

The Model Context Protocol (MCP) has quickly become a common way to connect AI agents to external systems, but a recurring design question is how those systems should be exposed to the model. A recent argument from the developer community holds that pointing AI agents at graphical user interfaces built for humans is an anti-pattern, and that MCP servers should instead surface machine-friendly interfaces such as APIs or command-line tools. The distinction matters because tool design appears to be one of the largest factors in whether an agent behaves reliably or fails in unpredictable ways.

The reasoning starts with what a GUI actually is. Graphical interfaces are optimized for human perception and motor control: they rely on spatial layout, color, icons, and the assumption that a person can visually locate an element and click it. When an AI agent is asked to drive that same interface, it typically has to capture screenshots, run a vision model to interpret pixels, infer coordinates, and then simulate mouse and keyboard input. Each of those steps introduces ambiguity and cost. A button that moves a few pixels, a modal dialog that appears unexpectedly, or a minor visual redesign can break an automation flow that previously worked, because the agent is reasoning about appearance rather than intent.

By contrast, machine-oriented interfaces expose structured, deterministic operations. An API endpoint or a CLI command takes named parameters and returns predictable, typed results, often as JSON or plain text. This is a far better match for how language models operate: they generate structured tool calls and consume structured responses. The article argues that routing work through these interfaces produces automation that is both more reliable and more efficient. It also tends to be more token-efficient, since a compact API response conveys the same information as a large screenshot would, without the overhead of image processing or lengthy visual descriptions.

This is precisely the gap MCP is meant to fill. MCP, introduced by Anthropic in late 2024 and since adopted across a range of clients and frameworks, standardizes how models discover and invoke tools, read resources, and use prompts. An MCP server sits between the agent and an underlying system, and the guidance here is that server authors should treat this boundary as an opportunity to design clean, purpose-built operations rather than to wrap a human workflow. Instead of a tool that says "open this page and click submit," a well-designed server would offer something like "create_order" or "query_records," exposing the capability directly and hiding the presentation layer entirely.

The recommendation should be read alongside the broader industry push toward so-called computer use, where models such as Anthropic's Claude and OpenAI's agent offerings can control a screen, move a cursor, and operate arbitrary desktop or browser applications. Those capabilities are genuinely useful when no programmatic interface exists, for example with legacy software or third-party websites that offer no API. The argument is not that GUI automation is worthless, but that it is likely a fallback of last resort rather than a default. When an API, SDK, or CLI is available, building the MCP tool on top of it is generally the sounder engineering choice.

There are practical trade-offs to weigh. Wrapping an internal API in an MCP server requires access, authentication handling, and ongoing maintenance as the underlying interface evolves. Some teams may not control the systems they need to automate, which is part of why visual agents exist at all. Good tool design also involves more than choosing an interface: clear naming, well-scoped parameters, sensible error messages, and appropriately granular operations all shape how well an agent can plan and recover from mistakes. Overly broad or ambiguous tools can undermine reliability even when they are API-based.

For developers building agent systems, the takeaway is to think of MCP as a design surface rather than a passthrough. Favoring machine-readable interfaces, reserving GUI control for cases where nothing else is available, and investing in clear tool definitions appear to be the more durable path. As the ecosystem matures, these conventions are likely to become part of the baseline expectations for how MCP servers are built and evaluated.