Multi-Agent Red-Blue Loop Auto-Generates and Proves Detection Rules

Detection-as-Code Foundry

An adversarial multi-agent detection-engineering workflow — LangGraph · Pydantic · FastMCP · Claude

Turn a threat (a CVE, a CISA KEV entry, an incident writeup, an ATT&CK technique) into a validated first-version detection — a proven starting point a detection team hardens further, not a claim of finished, forever-bulletproof coverage — plus three artifacts that ship together:

1. a Sigma detection rule that has survived a Blue-vs-Red adversarial loop,
2. a threat report with the round-by-round convergence history, and
3. a proof bundle — a positive corpus the rule fires on, a benign baseline it stays silent on, an adversarial mutation set, and a computed coverage label (real-coverage / brittle / broken) with FP/FN numbers and a reproducibility hash.

The thing nobody else produces is #3. A model can write you a Sigma rule; it cannot hand you a proof that the rule works, an adversary that provably evades the weak version, and an honest record of how the rule was hardened until it held.

A bounded, fixed adversary. Red commits to two real, documented attacks in round one and re-throws those same two against the hardened rule each round — the same "built once, can't move its goalposts" discipline as the fixed test corpus. real-coverage therefore means something concrete and honest: the first-version rule fires on every attack in the corpus, stays silent on benign look-alikes, and caught both of Red's real attacks. It is a validated baseline, not a promise of coverage against every future evasion — that is the detection team's job to extend.

Agents propose; the engine proves. Four role-specialized LLM agents (Threat Analyst, Blue Detection Engineer, Telemetry, Red-Team Adversary) do the creative work. Everything that decides truth — FP/FN evaluation, adversarial evasion, the coverage verdict, red-team verification, and the deployment gate — is a deterministic referee. Same input → same verdict → same reproducibility hash.

Two pipelines share this agent cast. The one described above proves — Blue authors, Red attacks, a deterministic referee decides. The Review Chain reviews — draft → Red Team → revision → SOC Lead → final rule, with every note that shaped it. Different question, different deliverable; pick per what you need.

📄 See real output first: examples/joomla-icagenda-cve-2026-48939/review-history.md — a live run on a CISA KEV entry, three rule versions, eight reviewer findings.

Run it

pip install -r requirements.txt          # or just run --evals on stdlib alone

# LIVE — any threat text; a real model authors it (needs GEMINI_API_KEY or ANTHROPIC_API_KEY)
python src/run.py "CVE-2026-45659 SharePoint deserialization RCE"
python src/run.py "T1218.011 rundll32 remote scriptlet execution via mshtml"

# CANNED — a fixture scenario, fully offline, no key (what the test suite drives)
python src/run.py --threat fixtures/certutil_t1105.json   # brittle→repair→proven

python src/run.py --corpus atomic         # positives from REAL Atomic Red Team commands
python src/run.py --max-rounds 3          # self-repair loop cap (default 3)
python src/run.py --evals                 # deterministic engine + proof + workflow tests
python src/run.py --approve alice         # human approves → gated deploy succeeds
python src/run.py --claude "..."          # force claude-opus-4-8 authoring (ANTHROPIC_API_KEY)
python src/run.py --gemini "..."          # force Gemini authoring (GEMINI_API_KEY)

There is no default scenario — the Foundry takes the threat you give it. A bare python src/run.py has nothing to work on and says so. Bundles are written to Outputs/<rule-slug>/ (rule.yml, report.md, proof.json).

（注：更多技术实现细节与完整 API 文档请查阅原项目 README）