Rohan Paul (@rohanpaul_ai) · 2026/10/16 01:06:13
自进化智能体框架优化：引入独立任务验证机制防止过拟合与能力退化
针对自改进智能体（Self-improving agents）修改自身提示词、记忆或代码时可能引发的过拟合与技能遗忘风险，提出基于“新鲜独立任务”的准入规则。该方法将进化算法映射至智能体框架优化中，要求新配置必须在未见过的测试集上显著优于当前版本方可保留。案例显示，经此流程优化的 GPT-5.4-nano 在 SpreadsheetBench 基准上得分超越仅做技能调优的 GPT-5.5，证实了严格验证机制对提升鲁棒性的价值。
报道全文原始报道全文
Rohan Paul (@rohanpaul_ai) · 2026-10-15 17
Self-improving agents should accept a harness edit only after it beats the current version on fresh, independent tasks.
It maps harness self-improvement methods onto evolutionary algorithms, from simple keep-the-best search to archives, Pareto selection, and bandit-guided budgets.
Self-improving agents now rewrite their harness: the prompts, memory, tools, and code around a fixed model. Accepted edits carry into later runs, so a bad edit can overfit, erase old skills, or spread to other users.
They sort these edits by what they change, from prompts to runtime code. Their admission rule: keep an edit only if it beats the current harness on new, independent tasks by a set margin.
In a case they compile from prior work, GPT-5.4-nano with an evolved harness and skills scored 77.58% on SpreadsheetBench. GPT-5.5 with skill tuning alone scored 76.20%, though costs were not matched.
If your agent edits its own setup, test each change on fresh tasks, and keep evaluators, permissions, and emergency stops out of its reach.
