Carnegie Mellon Benchmark Shows Opus 5.5 Agents Beat SGLang Production Systems

1 reports archived1 independent sourcesupdated 11/14/2026, 03:54:37

Synthesis & Latest Updates

1 Sources Cross-Validated

Carnegie Mellon University released Singularity-Bench, which tasks agents with six end-to-end engineering jobs spanning pre-training pipelines, inference engines and cluster orchestration, scored against production systems like SGLang and NVIDIA Dynamo on identical GPUs. Opus 5.5 with Claude Code reached a Singularity-Index of 1.25 (1 equals the human baseline) versus a 0.58 average for the other ten agents, and its self-built serving engine earned $11.52 per GPU-hour against $8.18 for SGLang.

LATEST/Carnegie Mellon University released Singularity-Bench, which tasks agents with six end-to-end engineering jobs spanning pre-training pipelines, inference engines and cluster orchestration, scored against production systems like SGLang and NVIDIA Dynamo on identical GPUs. Opus 5.5 with Claude Code reached a Singularity-Index of 1.25 (1 equals the human baseline) versus a 0.58 average for the other ten agents, and its self-built serving engine earned $11.52 per GPU-hour against $8.18 for SGLang.

Hourly heat curve

3 fully observed hours

Now

15.9

Peak

16.404:00

24h change

–

The line compares only sources observed throughout; gap hours are interpolated to keep the trend continuous.

Coverage timeline

Total 1 reports · Latest first

Rohan Paul (@rohanpaul_ai)T2Nov 14, 03:54·78 pts
Carnegie Mellon Benchmark Shows Opus 5.5 Agents Beat SGLang Production Systems
- Singularity-Bench covers six end-to-end engineering jobs including pre-training, inference engines and cluster orchestration, each compared against production systems on identical hardware
- Opus 5.5 with Claude Code scored a Singularity-Index of 1.25, while the other ten agents averaged just 0.58
- Its self-built serving engine earned $11.52 per GPU-hour, roughly 41% above SGLang's $8.18
View Related Reports & Analysis Original Source