NVIDIA's Nemotron 3.5 Super VL vision-language model is now available in Amazon SageMaker JumpStart, deployable via SageMaker Studio or the Python SDK. It pairs a 120B-parameter hybrid MoE language backbone with 12B active parameters and a C-RADIO vision encoder, supporting up to 1M-token context, multi-token prediction speculative decoding, and per-request reasoning configuration for document intelligence, GUI agents, and video QA.
Multi-Token Prediction (MTP)
多 Token 预测
Training and decoding mechanism predicting multiple future tokens per step, enabling speculative decoding.
Curated Intelligence
Baidu has open-sourced ERNIE 5.5 Flash Base, a Mixture-of-Experts foundation model with 280B total parameters and 14B activated per token, pretrained on 33.5T tokens and supporting up to 1M-token context in fine-tuning and inference. It claims same-size SOTA across knowledge, reasoning, math, code, multilingual and long-context benchmarks, surpassing DeepSeek-V4-Flash-Base and peers. Its efficiency-oriented innovations include Latent-Gated Sparse Attention, Multi-Router Latent MoE and the ECHO training objective that reuses the backbone's final layer.
Researchers identify a flaw in standard Multi-Token Prediction (MTP) training: the MTP cross-entropy gradient contains a low signal-to-noise residual that increasingly interferes with both next-token and multi-token learning at scale. They introduce a KL gradient-recovery trick that exactly recovers the KL self-distillation term from cross-entropy gradients via straight-through gradient routing, yielding Soft-MTP—a one-line code change requiring no vocabulary-sized logit matrices and compatible with fused cross-entropy kernels. At 1.2B scale it improves math and code reasoning after fine-tuning and matches or exceeds direct KL minimization on most metrics.
JetBrains released the open-source Mellum2.1 model, significantly enhancing agentic coding capabilities by expanding reinforcement learning as a core training phase. The model achieves nearly double the high-load inference throughput of Qwen3.5-9B on H200 GPUs and supports local deployment for code privacy.
The vLLM team has released vllm-metal, bringing vLLM's scheduler, paged KV cache, and OpenAI-compatible server to Apple Silicon. By leveraging MLX and Metal for execution, it addresses concurrency issues like Time to First Token (TTFT) and memory growth in local Mac inference. The current v0.29.0 version supports Homebrew installation and introduces batched multi-token prediction (MTP) along with GGUF model support.
vLLM officially announced Day-0 support for the 1T-parameter multimodal model TML Inkling from Thinking Machines Lab, covering both BF16 and NVFP4 variants. The integration optimizes novel architectural components like relative attention, short convolution, and shared expert sinks, achieving up to 380 tok/s on 4 GB200 GPUs with Multi-Token Prediction (MTP) while maintaining full feature parity including LoRA and parallelism.
Google introduces a method to retrofit Multi-Token Prediction (MTP) onto existing 'frozen' Gemini Nano models for Pixel devices. This approach addresses mobile autoregressive bottlenecks by improving inference speed and energy efficiency without retraining the base model.