Termsslug: multi-token-prediction-mtp

Multi-Token Prediction (MTP)

多 Token 预测

Training and decoding mechanism predicting multiple future tokens per step, enabling speculative decoding.

Curated Intelligence

7 items · Page 1/1
AWS Machine Learning BlogT1·Models
62 pts
NVIDIA Nemotron 3.5 Super VL Lands in Amazon SageMaker JumpStart

NVIDIA's Nemotron 3.5 Super VL vision-language model is now available in Amazon SageMaker JumpStart, deployable via SageMaker Studio or the Python SDK. It pairs a 120B-parameter hybrid MoE language backbone with 12B active parameters and a C-RADIO vision encoder, supporting up to 1M-token context, multi-token prediction speculative decoding, and per-request reasoning configuration for document intelligence, GUI agents, and video QA.

量子位官网T2·Models
78 pts
Baidu Open-Sources ERNIE 5.5 Flash Base, a 280B MoE Model Topping Same-Size Benchmarks

Baidu has open-sourced ERNIE 5.5 Flash Base, a Mixture-of-Experts foundation model with 280B total parameters and 14B activated per token, pretrained on 33.5T tokens and supporting up to 1M-token context in fine-tuning and inference. It claims same-size SOTA across knowledge, reasoning, math, code, multilingual and long-context benchmarks, surpassing DeepSeek-V4-Flash-Base and peers. Its efficiency-oriented innovations include Latent-Gated Sparse Attention, Multi-Router Latent MoE and the ECHO training objective that reuses the backbone's final layer.

Hacker News AIT2·Models
74 pts
Soft-MTP Fixes Gradient Interference in Multi-Token Prediction

Researchers identify a flaw in standard Multi-Token Prediction (MTP) training: the MTP cross-entropy gradient contains a low signal-to-noise residual that increasingly interferes with both next-token and multi-token learning at scale. They introduce a KL gradient-recovery trick that exactly recovers the KL self-distillation term from cross-entropy gradients via straight-through gradient routing, yielding Soft-MTP—a one-line code change requiring no vocabulary-sized logit matrices and compatible with fused cross-entropy kernels. At 1.2B scale it improves math and code reasoning after fine-tuning and matches or exceeds direct KL minimization on most metrics.

vLLM 官方博客T1·Tools
68 pts
vLLM Launches vllm-metal for Concurrent Inference on Apple Silicon

The vLLM team has released vllm-metal, bringing vLLM's scheduler, paged KV cache, and OpenAI-compatible server to Apple Silicon. By leveraging MLX and Metal for execution, it addresses concurrency issues like Time to First Token (TTFT) and memory growth in local Mac inference. The current v0.29.0 version supports Homebrew installation and introduces batched multi-token prediction (MTP) along with GGUF model support.

vLLM 官方博客T1·Tools
68 pts
vLLM Adds Day-0 Support for TML Inkling: 1T Multimodal Model with Relative Attention and MTP Acceleration

vLLM officially announced Day-0 support for the 1T-parameter multimodal model TML Inkling from Thinking Machines Lab, covering both BF16 and NVFP4 variants. The integration optimizes novel architectural components like relative attention, short convolution, and shared expert sinks, achieving up to 380 tok/s on 4 GB200 GPUs with Multi-Token Prediction (MTP) while maintaining full feature parity including LoRA and parallelism.