Termsslug: cuda-graphs

CUDA Graphs

CUDA 图

Technology capturing GPU operation sequences as static graphs to reduce launch overhead

Curated Intelligence

2 items · Page 1/1
vLLM 官方博客T1·Tools
68 pts
vLLM Optimizes DeepSeek-V4.1-Flash: 5.3x Agentic Throughput via New Kernels and CUDA Graphs

Inferact and the vLLM community achieved a 5.3x throughput improvement for DeepSeek-V4.1-Flash in agentic scenarios within three weeks of its release. Key optimizations include integrating DeepSeek's new kernels (MegaAttention, Mega-mHC) and implementing SWA bounded replay with CUDA graphs to reduce Time-To-First-Token by ~30%. Aggressive kernel fusion and parallelization further unlocked the memory efficiency of the model's causal encoder-decoder architecture.

vLLM 官方博客T1·Tools
68 pts
vLLM-Omni Unveils Multi-Stage Serving for Qwen3-Omni: Pipeline and Async Optimizations Cut Audio Latency

The vLLM-Omni project details its multi-stage serving architecture for the Qwen3-Omni multimodal model, utilizing a pipeline of Thinker, Talker, and Code2Wav stages. By implementing stage-level batching, CUDA Graphs, and asynchronous chunk handoffs, the system mitigates decoding stalls in high-concurrency scenarios and significantly reduces Time to First Audio Packet (TTFP). This work provides a reusable performance optimization paradigm for deploying omni-modality models.