Toolsslug: swe-bench-tool

SWE-bench

Benchmark evaluating models on resolving real-world GitHub software engineering issues.

Curated Intelligence

1 items · Page 1/1
The DecoderT2·Agents
78 pts
AI Coding Agents Complete Under 10% of Simulated Workplace Tasks

A new experiment places frontier coding agents such as Claude Opus 5.5 and GPT-6 Astra inside a simulated software company with colleagues, project boards, and chat histories to test them as digital employees. Despite near-saturated scores on isolated benchmarks like SWE-bench, the agents fully completed fewer than one in ten tasks, exposing a wide gap between benchmark performance and real workplace collaboration.