亚马逊新研究:环境变更后陈旧智能体技能反而有害
亚马逊最新论文指出,智能体保存的技能(如文件路径、字段名、命令)在应用或 API 变更后会静默失效,反而拖累任务表现。在 Terminal-Bench 2 上,原本为 Opus 提升 4.7 个百分点的技能在环境变化后使通过率比无技能低 11.4 个百分点;其选择性校验与修复方案将 Haiku 税表字段准确率从 30.5% 提升至 97.5%。
报道正文
Rohan Paul (@rohanpaul_ai) · 2026-11-19 19
New Amazon paper finds that saved agent skills can hurt once the environment changes, and checking and repairing them selectively restores their value.
Agent skills store past experience, like file paths, field names, or commands, so agents can skip work later. When an app or API changes, some of that stored detail goes stale without warning.
They tested Claude Opus 4.7 and Haiku 4.5 on 4 settings, including Terminal-Bench 2 and IRS tax forms, before and after a change. Their fix replays saved checks witho1ut model calls and repairs or rebuilds only the conflicting skill content.
On Terminal-Bench 2, a skill that had added 4.7 points for Opus cut its pass rate 11.4 points below no skill after the change. Maintenance lifted Haiku's tax-form field accuracy from 30.5% with the stale skill to 97.5%.
Check saved agent skills against the current environment before reuse, instead of waiting for tasks to fail.
