Termsslug: tokens-per-second

Tokens Per Second

TPS

A metric indicating how many text units an LLM generates per second.

Curated Intelligence

3 items · Page 1/1
量子位官网T2·Models
42 pts
OpenAI Launches 'Crazy 28 Days': GPT-6 Inference Speed Up 50%

OpenAI's Codex lead launched a 28-day intensive update sprint, delivering a ~50% inference speed boost for GPT-6 Astra and Sol on day one. Despite the performance optimization, community reaction was largely sarcastic, viewing it as PR damage control following prior latency issues and commercialization controversies.

vLLM 官方博客T1·Tools
68 pts
vLLM Optimizes Qwen3.5 Hybrid Architecture: Achieving 25K TPS/GPU on GB200 NVL72 Systems

The vLLM community has optimized serving for Qwen3.5's hybrid attention architecture (combining full-attention and GDN layers), achieving over 25K TPS per GPU on NVIDIA GB200 NVL72 systems. By addressing heterogeneous state transfer and accelerating GDN computation on Blackwell GPUs, this solution significantly boosts the performance ceiling of disaggregated serving.