Carnegie Mellon Benchmark Shows Opus 5.5 Agents Beat SGLang Production Systems
Carnegie Mellon University released Singularity-Bench, which tasks agents with six end-to-end engineering jobs spanning pre-training pipelines, inference engines and cluster orchestration, scored against production systems like SGLang and NVIDIA Dynamo on identical GPUs. Opus 5.5 with Claude Code reached a Singularity-Index of 1.25 (1 equals the human baseline) versus a 0.58 average for the other ten agents, and its self-built serving engine earned $11.52 per GPU-hour against $8.18 for SGLang.
Related Entities
Full Report
Rohan Paul (@rohanpaul_ai) · 2026-11-13 19
New Carnegie Mellon paper finds that Opus 5.5 + Claude Code built a pre-training pipeline, an inference engine, and a cluster orchestrator that matched or beat production tools like SGLang and NVIDIA Dynamo on the same GPUs.
Singularity-Bench gives agents 6 end-to-end jobs, from pre-training a model to building a serving engine. Each result is scored against a production system, such as SGLang or Claude Code, on the same GPUs.
Opus 5.5 scored a Singularity-Index of 1.25, where 1 equals the human baselines, while the other 10 agents averaged 0.58. Its serving engine earned 11.52 USD per GPU-hour against 8.18 for SGLang.
