vLLM Adds Day-0 Support for Kimi K3.1 Pro with Sparse Attention

Moonshot AI has officially open-sourced Kimi K3.1 Pro, its most powerful native multimodal agent model to date. Featuring a Mixture-of-Experts (MoE) architecture with 2.8T total parameters and 104B active parameters, the model supports a massive 1 million token context window. Through large-scale joint reinforcement learning post-training, it demonstrates significant improvements in complex tasks such as coding, 3D generation, and financial research.

vLLM immediately announced Day-0 support for the model, addressing inference bottlenecks associated with ultra-large parameters and long contexts. Kimi K3.1 Pro introduces Sparse Attention (DSA) within its MLA layers, reducing decoded byte reads by approximately 75%. By leveraging techniques like Token Sharding Prefill and Decoding Context Parallelism (DCP), vLLM achieves a 1.69x speedup in time-to-first-token for 340K-length texts and boosts attention module speed by 1.3-1.5x, significantly enhancing both efficiency and throughput.

This development marks a new milestone in engineering capabilities for open-source models combining ultra-large MoE architectures with extended contexts. For developers, Day-0 support enables immediate deployment of high-performance inference services, lowering technical barriers. The successful validation of sparse attention optimization provides an efficient architectural reference for handling million-token documents and complex agent tasks, potentially accelerating industrial applications in long-document analysis.