Rohan Paul (@rohanpaul_ai) · 10/12/2026, 14:11:41
AWS and HSBC Introduce Double Ratchet: Self-Evolving Graders Enable Unsupervised Agent Improvement from 10 Samples
A joint paper by AWS and HSBC introduces the 'Double Ratchet' framework, eliminating the need for human-written graders in self-improving agents. By co-evolving the grader with agent skills using only 10 labeled examples to build rule-based checks, the method achieves training performance comparable to ground truth or rubrics (retaining 88-110% of gains). This significantly lowers the barrier for unsupervised agent self-improvement.
SOURCE COVERAGEFull Original Report
Rohan Paul (@rohanpaul_ai) · 2026-10-12 06
Self-improving agents have always needed a human to write the grader.
A new AWS + HSBC paper removes that step: the grader evolves alongside the agent from 10 labeled examples and trains it about as well as ground truth.
Double Ratchet evolves the grader alongside the agent's skills, building it from small, mostly rule-based checks for 1 failure type each. A grader version is chosen for agreeing with 10 labeled examples, never for raising the agent's score.
It kept 88% to 110% of the gain that ground truth or a rubric gave the same skill loop, across code, SQL, and reports. With the safeguards removed, the grader passed 97% to 100% of outputs yet trained skills as well.
