Small AI models just got a surprising boost from a very old game.
MIT researchers used a Battleship-style setup to test whether AI agents can improve how they gather information before making a move. The result was a sharp jump in performance for smaller systems, including one model that went from rarely beating humans to winning most of its games after researchers changed how it searched the board.
That shift goes straight at one of the biggest weaknesses in today’s AI agents. They’re often asked to handle tasks where the answer depends on details they don’t have yet. MIT’s work suggests better question planning can make a cheaper model act far more capable.
How much smarter did it get
MIT’s test used a version of Battleship built around natural-language questions. One AI agent played the role of the teammate trying to locate hidden ships, while another had access to the board and answered.
The biggest jump came from Llama 4 Scout. MIT said the smaller model beat human players in only 8% of games at first. After researchers added a more deliberate inference strategy, it beat humans 82% of the time and outpaced a larger frontier model while operating at about 1% of the cost.
That’s the number to watch if you care about AI costs. The model didn’t win by getting larger, but won by choosing sharper questions and making better use of each answer.
Why does Battleship help AI learn
Battleship works as a test because it forces an AI agent to act with limited information. It can’t see the whole board, so every question has to narrow the search and set up the next move.
That maps neatly onto practical AI tools. A support bot, research assistant, or planning agent often needs to ask follow-ups before it can help. When that process breaks down, the model can miss a key detail, repeat itself, or make a recommendation too early.
The MIT approach puts pressure on that weak spot. It measures whether an agent can gather the right information before producing an answer.
Where could this go next
The harder test is whether the same approach works beyond games. Battleship is controlled, which makes it easier to score than open-ended agent workflows in search, customer support, or workplace software.
Still, the direction is worth watching. If smaller models learn to ask sharper questions before acting, companies could build cheaper AI tools that feel more capable in everyday use.
The next milestone is transfer from the game board to real work. A task with unclear instructions, missing files, and a rushed user will be much harder to solve.