Watch: Google's Approach to Recursive Self-Improvement

Optimizing an artificial intelligence agent can be slow and incredibly expensive. Every time you try a new exploration strategy, it requires live model calls and real code executions. Researchers from Google, Google DeepMind, and the Universities of Maryland and Virginia have developed a way to bypass this bottleneck. Their method is called Dream-RSI. Instead of running expensive live trials, Dream-RSI uses the agent’s own history as an exact, offline simulator. When the agent explores a task, it records every attempt, code execution, and score into a tree of data. Because the outcomes of these past runs are already saved, the system can replay them instantly. In the dreaming phase, a large language model rewrites the agent’s exploration policy. The system then tests thousands of these new candidate strategies against the historical data at zero cost. Since the outcomes are cached on disk, evaluating a new strategy requires only simple memory lookups. No new code is executed, and no live model calls are made. The best-performing policy is then redeployed for the next real-world run. This approach drastically cuts down on computing costs. In benchmark tests, Dream-RSI matched or outperformed state-of-the-art baselines. It used up to one hundred and sixty-two times fewer agent calls for algorithm engineering, roughly halved the generations needed for graphics processing unit kernel engineering, and saved over fifty times the budget on mathematical optimization. By turning history into a perfect simulator, Dream-RSI makes AI search strategies both smarter and cheaper to find.