Toolsslug: jittests

JiTTests

Meta's just-in-time testing system where LLMs generate tests expected to fail on a diff to expose behavioral differences.

Curated Intelligence

1 items · Page 1/1
Marktechpost (@Marktechpost)T2·Tools
72 pts
Meta Unveils JiTTests Where LLMs Write Failing Tests to Catch Bugs

Meta introduced JiTTests, which has LLMs generate tests expected to fail against a code diff, proving the change altered behavior before it lands; an LLM judge ensemble plus a rule-based filter called RubFake then decides which failures merit engineer attention. In H1 2026 it caught 74 privacy SEVs at the diff stage, cut privacy SEV occurrence by 20%, and drew a roughly 60% engineer-reported true-positive rate across 589 engineers. Intent-aware tests showed a 6.4% weak-catch rate per test versus 0.8% for Meta's earlier ACH hardening, though the initial paper deployment confirmed only 8 of 41 reported catches and all production figures are self-reported.