Gemini now coaches SIMA 2

DeepMind layered its multimodal Gemini model on top of SIMA 2, creating an agent that follows voice, text, and sketch prompts across open-ended games.

New capabilities

• Better spatial reasoning: navigate large maps and remember objectives across sessions.
• Multimodal input: combine speech, text, and simple drawings to clarify intent.
• Strategy coaching: suggest next actions while the player keeps control.

Why it matters

• Shows how foundation models can generalize across diverse 3D worlds without retraining per game.
• Opens a path to assistant-style copilots for creative tools and simulation training.

Cautions

• Safety: agents must avoid griefing behaviors or exploiting game bugs.
• Privacy: voice and gameplay streams need clear consent and retention limits.
• Reliability: open worlds are unpredictable; fail-safes and handoff to humans remain key.

Takeaway: Gemini-powered SIMA 2 is a glimpse of cross-world assistants; the real test will be keeping them safe, steerable, and respectful of player agency.