Termsslug: operator-fusion

Operator Fusion

算子融合

Combining multiple consecutive operations into a single kernel to reduce memory access and launch overhead

Curated Intelligence

4 items · Page 1/1
IT之家 · 智能时代T2·Infra
68 pts
NVIDIA Optimizes Kimi K3 Inference Kernel for 1.68x Speedup

NVIDIA achieved a 1.68x inference speedup for the 2.8T-parameter Kimi K3 model on GB300 NVL72 hardware via upstream vLLM optimizations, reaching 423 tok/s/user. The breakthrough relies on a massive 12,000-line Megakernel written in NVIDIA's new Cute DSL, demonstrating the future of AI-assisted generation of advanced compute kernels.

vLLM 官方博客T1·Infra
68 pts
vLLM Uses AI Agents to Optimize Kimi K3 Inference on GB300

Moonshot AI leveraged agent-assisted development in vLLM to build a unified CuTe DSL decode path for Kimi K3. The optimization achieves over 840 tokens/s at batch 1 on NVIDIA GB300 clusters and boosts throughput by 1.5–2x in tested configurations, marking a significant step in using AI to write low-level GPU kernels.

Hugging Face BlogT1·Tools
48 pts
PyTorch Profiling: From nn.Linear to Fused MLP

Hugging Face releases the second part of its PyTorch profiling series, demonstrating how to use profiler traces to optimize standard nn.Linear layers into fused MLPs. The article provides a concrete case study on identifying computational bottlenecks and implementing operator fusion for improved inference efficiency.

Qwen 官方博客 (通义)T1·Infra
78 pts
Qwen Open-Sources FlashQLA: 2-3x Speedup for GDN Linear Attention

The Qwen team has open-sourced FlashQLA, a high-performance linear attention library built on TileLang and optimized for Gated Delta Networks (GDN). By leveraging gate-driven intra-GPU sequence parallelism and hardware-friendly algebraic rewrites, it achieves 2-3x forward and 2x backward speedups on NVIDIA Hopper GPUs, significantly reducing training and inference costs for long-context models.