DeepSeek V4.1 Flash Gains Up to 1.5x Per-Agent Speed on GB300 NVL72

1 reports archived1 independent sourcesupdated 11/7/2026, 04:33:47

Synthesis & Latest Updates

1 Sources Cross-Validated

Artificial Analysis' AA-AgentPerf benchmark shows DeepSeek V4.1 Flash on GB300 NVL72 rising from 596 to 848 tok/s for a single agent (1.42x), with roughly 1.6x throughput per GPU at ~80 tok/s per agent. The gains stem from a newer vLLM release with GB300-specific kernels, FlashInfer autotuning, fuller CUDA graph coverage, tuned speculative decoding, and co-locating prefill and decode on the same GPUs in NVIDIA's updated Dynamo + vLLM stack.

LATEST/Artificial Analysis' AA-AgentPerf benchmark shows DeepSeek V4.1 Flash on GB300 NVL72 rising from 596 to 848 tok/s for a single agent (1.42x), with roughly 1.6x throughput per GPU at ~80 tok/s per agent. The gains stem from a newer vLLM release with GB300-specific kernels, FlashInfer autotuning, fuller CUDA graph coverage, tuned speculative decoding, and co-locating prefill and decode on the same GPUs in NVIDIA's updated Dynamo + vLLM stack.

Hourly heat curve

2 fully observed hours

Fewer than 3 fully observed hours; trend not plotted yet.

Coverage timeline

Total 1 reports · Latest first

Artificial Analysis (@ArtificialAnlys)T1Nov 7, 04:33·72 pts
DeepSeek V4.1 Flash Gains Up to 1.5x Per-Agent Speed on GB300 NVL72
- Single-agent speed reaches 848 tok/s, up 1.42x from 596 tok/s a month earlier
- Throughput per GPU is about 1.6x higher at roughly 80 tok/s per agent
- Gains come from software-layer work: new vLLM kernels, FlashInfer autotuning, fuller CUDA graph coverage and tuned speculative decoding
View Related Reports & Analysis Original Source