浙大研究：弱模型可当裁判训练出更强智能体
浙江大学新论文提出，一个自身任务成功率仅 42.9% 的决策模型 JEV，在事后逐动作评审另一智能体的轨迹后，可将其在 ALFWorld 上的成功率从 72.8% 提升至 82.9%。该方法把评审模型给出的逐步评分并入 GRPO 训练，绕开了传统结果奖励对每一步同等归因、以及逐步标注需环境标签或 LLM 裁判的高成本问题。作者建议按“评审记录轨迹的能力”而非“自身执行能力”来挑选步骤裁判。
报道正文
Rohan Paul (@rohanpaul_ai) · 2026-11-14 18
AI model that is itself bad at a task can still train another AI to do it well.
New paper shows that a decision model called JEV, an AI model built to pick between options instead of writing text, completes only 42.9% of simulated household tasks on its own, yet by reviewing another AI’s moves afterward it raised that AI’s success from 72.8% to 82.9%.
New Zhejiang University paper shows that an AI model that is bad at a task can still train another AI to do it well, by reviewing each of its moves after the attempt ends.
Agent RL usually rewards only the final outcome, so every step gets the same credit. Scoring each step usually needs per-environment labels or an LLM judge at every action.
They tested decision models, which return a choice with probabilities instead of text, as actors and as step graders on ALFWorld and WebShop. As a grader, the model sees the whole run and scores whether each action helped, and those scores are added to GRPO training.
The decision model JEV completes only 42.9% of ALFWorld tasks when acting. As a grader, it lifts a Qwen2.5-1.5B policy to 82.9% success, up from 72.8% with GRPO outcome rewards alone.
Choose step graders by how well they judge recorded runs, not how well they act, and ask about each action, not the outcome.
