Toolsslug: singularity-bench

Singularity-Bench

Agent benchmark for end-to-end AI engineering tasks, scored against production systems via the Singularity-Index.

Curated Intelligence

1 items · Page 1/1
Rohan Paul (@rohanpaul_ai)T2·Agents
78 pts
Carnegie Mellon Benchmark Shows Opus 5.5 Agents Beat SGLang Production Systems

Carnegie Mellon University released Singularity-Bench, which tasks agents with six end-to-end engineering jobs spanning pre-training pipelines, inference engines and cluster orchestration, scored against production systems like SGLang and NVIDIA Dynamo on identical GPUs. Opus 5.5 with Claude Code reached a Singularity-Index of 1.25 (1 equals the human baseline) versus a 0.58 average for the other ten agents, and its self-built serving engine earned $11.52 per GPU-hour against $8.18 for SGLang.