HOT RADAR
toolsHEAT 17.1°

AI CURATED HOT STORY · 11/16/2026

Meta Unveils JiTTests Where LLMs Write Failing Tests to Catch Bugs

1 reports archived1 independent sourcesupdated 11/16/2026, 18:41:45
Synthesis & Latest Updates
1 Sources Cross-Validated

Meta introduced JiTTests, which has LLMs generate tests expected to fail against a code diff, proving the change altered behavior before it lands; an LLM judge ensemble plus a rule-based filter called RubFake then decides which failures merit engineer attention. In H1 2026 it caught 74 privacy SEVs at the diff stage, cut privacy SEV occurrence by 20%, and drew a roughly 60% engineer-reported true-positive rate across 589 engineers. Intent-aware tests showed a 6.4% weak-catch rate per test versus 0.8% for Meta's earlier ACH hardening, though the initial paper deployment confirmed only 8 of 41 reported catches and all production figures are self-reported.

LATEST/Meta introduced JiTTests, which has LLMs generate tests expected to fail against a code diff, proving the change altered behavior before it lands; an LLM judge ensemble plus a rule-based filter called RubFake then decides which failures merit engineer attention. In H1 2026 it caught 74 privacy SEVs at the diff stage, cut privacy SEV occurrence by 20%, and drew a roughly 60% engineer-reported true-positive rate across 589 engineers. Intent-aware tests showed a 6.4% weak-catch rate per test versus 0.8% for Meta's earlier ACH hardening, though the initial paper deployment confirmed only 8 of 41 reported catches and all production figures are self-reported.

Hourly heat curve

2 fully observed hours
Fewer than 3 fully observed hours; trend not plotted yet.

Coverage timeline

Total 1 reports · Latest first
  1. Marktechpost (@Marktechpost)T2·72 pts
    • The core idea is having LLMs generate tests expected to fail on a diff, using executable evidence of behavior change instead of static review
    • H1 2026 caught 74 privacy SEVs at the diff stage, cut privacy SEV occurrence 20%, and saw a ~60% engineer-reported true-positive rate across 589 engineers
    • Intent-aware tests hit a 6.4% weak-catch rate per test versus 0.8% for ACH hardening, but only 8 of 41 early reported catches were confirmed and all production figures are self-reported