SIMA 2: 仮想3D世界で推論・学習する Gemini 搭載エージェント SIMA 2: An Agent that Plays, Reasons, and Learns With You in Virtual 3D Worlds

Google DeepMind has introduced SIMA 2, the next iteration of its generalist agent for 3D virtual worlds. Where the original SIMA focused on following short natural-language instructions across commercial video games, SIMA 2 integrates Gemini as a reasoning backbone, turning the system from an instruction-follower into something closer to a collaborative game-playing partner that can plan, explain its actions, and learn on its own.

Like its predecessor, SIMA 2 perceives only the rendered pixels of a game and acts through the same keyboard-and-mouse interface a human would use. That constraint is deliberate: it forces the agent to generalize across engines and genres rather than relying on privileged APIs. The new version was trained across a broad portfolio of titles and research environments, and DeepMind reports significant gains on long-horizon, multi-step tasks, with performance on complex goals approaching human-level in evaluated games.

The Gemini integration unlocks several qualitative jumps. SIMA 2 can interpret abstract instructions, respond to sketches or emoji prompts, and follow directions in languages it was not explicitly trained on. It can also narrate its reasoning, which is useful both for user trust and for debugging agent behavior. Perhaps more importantly, DeepMind describes a self-improvement loop in which Gemini proposes tasks and reward signals in new environments, letting SIMA 2 bootstrap competence in unfamiliar games without fresh human demonstrations. The team also shows the agent operating inside procedurally generated worlds from Genie, hinting at a tighter coupling between DeepMind's world-model research and its embodied-agent line.

The broader context matters. Generalist agents that act through perception and motor-like outputs have become a crowded research frontier: OpenAI's VPT learned Minecraft from video, NVIDIA's Voyager used LLM-driven curricula in the same game, Anthropic's Computer Use and Google's own Project Mariner extend the idea to desktop and browser interfaces. SIMA 2 sits somewhere between these efforts, leaning on rich 3D simulation as a training ground for skills DeepMind has openly described as relevant to robotics and embodied AI. Games are cheap, diverse, and safe; a policy that can take a vague human goal in an unfamiliar 3D world and decompose it into grounded actions is exactly the kind of capability that real-world robots will eventually need.

There are caveats. Pixel-to-action play in games still differs from physical manipulation, where sensing noise, contact dynamics, and safety constraints dominate. Self-generated rewards via an LLM can drift or reward-hack, and DeepMind's own reporting acknowledges that long-horizon reliability remains imperfect. The release is positioned as a limited research preview rather than a product, with access gated while the team studies safety and alignment behaviors.

Still, SIMA 2 is a notable marker. It suggests that combining a strong multimodal reasoner like Gemini with an action-capable policy and a steady supply of simulated worlds may be a viable recipe for agents that keep getting better with experience. Whether that recipe transfers cleanly out of the game window is the open question the next year of research is likely to answer.