The DecoderT2·78 pts
- Frontier coding agents near saturation on isolated benchmarks like SWE-bench but fully complete under 10% of tasks in a simulated workplace
- The experiment adds colleagues, project boards, and chat histories to test end-to-end closure rather than single-shot code generation
- Results indicate a systematic gap between current benchmarks and real-world software engineering capability