Termsslug: p90-time-to-first-token

P90 Time-to-First-Token

P90 首 token 延迟

The 90th-percentile wait from request dispatch to first output token, capturing tail response latency.

Curated Intelligence

1 items · Page 1/1
新智元 (微信公众号)T2·Agents
78 pts
Claude Writes Its Own Inference Engine, Cutting First-Token Latency to 0.48 Seconds

Researchers from CMU, NVIDIA and other institutions posed a task in the Singularity-Bench preprint: give Claude Opus 5.5 base server code, four GB300 GPUs and 12 hours, with no internet access, to build an inference engine using Claude Code. Replaying 16 concurrent real coding sessions, the agent-written engine reached roughly 0.48s P90 time-to-first-token versus about 12s for the mature engine SGLang v0.5.20, moving the question of whether agents can build agents from speculation to reproducible engineering evidence.