Google and Johns Hopkins Unveil Pro2 Agent That Tracks Bottlenecks Before Coding

Original title: New Google and Johns Hopkins paper shows that AI research agents do better when they track what limi...
78AI Score

Google and Johns Hopkins researchers introduce Pro2 Agent, which has ML research agents first build a 'Problem State' capturing suspected bottlenecks, candidate fixes and confidence levels, then run single-variable experiments to update beliefs before committing code changes. It reaches a 76.0% medal rate across all 75 MLE-Bench tasks within 48 hours, the highest reported, and medals on 57.6% of MLE-Bench Lite tasks with its first solution versus 28.8% for MARS.

Full Report

Rohan Paul (@rohanpaul_ai) · 2026-11-20 19

New Google and Johns Hopkins paper shows that AI research agents do better when they track what limits performance, not just which code scored best.

Agents that do ML research usually loop on code: change it, score it, keep the best. On hard tasks they can get stuck tweaking a solution that is wrong at its core.

Pro2 Agent first builds a "Problem State": the bottlenecks it suspects, candidate fixes from the literature, and how confident it is in each. It then runs single-variable experiments that update those beliefs before committing changes.

On all 75 MLE-Bench tasks, it reached a 76.0% medal rate with 48 hours, the highest reported, using Gemini 3.5 Flash for coding. On MLE-Bench Lite, its first solution already medaled on 57.6% of tasks, against 28.8% for MARS.

If you build research agents, make them write down and update what is blocking progress, and test 1 change per experiment.

推文配图