HOT RADAR
agentHEAT 11.9°

AI CURATED HOT STORY · 11/22/2026

AI Coding Agents Complete Under 10% of Simulated Workplace Tasks

1 reports archived1 independent sourcesupdated 11/22/2026, 23:15:48
Synthesis & Latest Updates
1 Sources Cross-Validated

A new experiment places frontier coding agents such as Claude Opus 5.5 and GPT-6 Astra inside a simulated software company with colleagues, project boards, and chat histories to test them as digital employees. Despite near-saturated scores on isolated benchmarks like SWE-bench, the agents fully completed fewer than one in ten tasks, exposing a wide gap between benchmark performance and real workplace collaboration.

LATEST/A new experiment places frontier coding agents such as Claude Opus 5.5 and GPT-6 Astra inside a simulated software company with colleagues, project boards, and chat histories to test them as digital employees. Despite near-saturated scores on isolated benchmarks like SWE-bench, the agents fully completed fewer than one in ten tasks, exposing a wide gap between benchmark performance and real workplace collaboration.

Coverage timeline

Total 1 reports · Latest first
  1. The DecoderT2·78 pts
    • Frontier coding agents near saturation on isolated benchmarks like SWE-bench but fully complete under 10% of tasks in a simulated workplace
    • The experiment adds colleagues, project boards, and chat histories to test end-to-end closure rather than single-shot code generation
    • Results indicate a systematic gap between current benchmarks and real-world software engineering capability