vLLM 官方博客T1·Infra
78 pts
The vLLM team deployed DeepSeek-V4.1-Flash on a single NVIDIA Vera Rubin NVL72 rack, reaching 1.7M tokens/s per GPU and 387 tokens/s per user, a 58x gain over GB300 NVL72, with peak throughput of 2.52M tokens/s. The speedup stems from kernels tuned for Rubin's split memory and FP4 tensor cores, specialized low-latency decode kernels, and rack-wide KV cache and Engram tables shared over NVLink.