Alibaba released Qwen4-Max, a 2.4T-parameter sparse MoE flagship that activates only 71B parameters per token at $1.5 per million input tokens. Its rebuilt sparse MoE stack uses three Gated DeltaNet layers to compress history for each Qwen Sparse Attention layer, a lightweight indexer for context selection, plus four gated residual branches and 328B N-gram embedding parameters. It tops 4 of 16 benchmarks but trails Claude Opus 5.5 and Kimi K3.1 on coding and terminal tasks, a clear trade-off.
Sparse MoE
稀疏混合专家
A model structure that improves training and inference efficiency by dynamically routing inputs to activate only a subset of expert networks.
Curated Intelligence
Dutch chip startup Axelera AI has disclosed the architecture of Titania, its third-generation AIPU targeting data center inference, with mass production slated for Q3 2027 and over $300 million in orders already booked. Optimized for sparse MoE models, the chip delivers 2.5 PFLOPS at FP8, and a single 4U eight-card server can run a 1.6T MoE model locally, with the company claiming 7x per-watt throughput versus current data center servers.
Baidu released ERNIE 5.5 Flash Base, a 280B-total, 14B-active sparse MoE model shipped directly from pre-training with no mid- or post-training. Efficiency is the design focus: 37 of 43 layers use Heavily Compressed Attention caching one entry per 128 tokens, multiple routers select top-10 of 512 experts, and the final layer doubles as a speculative-decoding draft model. Baidu self-reports gains over DeepSeek-V4-Flash-Base on MMLU-Pro and HumanEval+, but trails on HELMET-Rerank, and all numbers are self-reported.
Baidu released ERNIE 5.5 Flash Base, a sparse Mixture-of-Experts language model with 280B total parameters and 14B activated per token, pre-trained from scratch on 33.5T tokens with a 128K training context and 1M-token support for fine-tuning and inference. The release is the end-of-pre-training checkpoint before any mid- or post-training, positioned as a common starting point for domain adaptation, instruction tuning and reinforcement-learning research.
Baidu has open-sourced the pretraining-endpoint weights of ERNIE-5.5-Flash-Base, a sparse MoE language model with 280B total parameters and 14B activated per token, pretrained from scratch on 33.5T tokens with up to 1M-token context for fine-tuning and inference. The weights are released without mid- or post-training, positioned as a shared starting point for domain adaptation, instruction tuning and RL research, with the post-trained ERNIE 5.5 Flash to follow.
Baidu has released and open-weighted ERNIE 5.5 Flash Base, a sparse MoE language model with 280B total parameters and 14B activated per token, pretrained from scratch on 33.5T tokens with a 128K training context that extends to 1M tokens during fine-tuning and inference. Positioned as a general-purpose foundation for downstream adaptation, it ships with vLLM and SGLang ecosystem support and targets efficiency and cost balance for long-running agent workloads.
Meta released Muse Spark Open, its strongest US open-weight model featuring a sparse MoE architecture with million-token context and multimodal input, optimized for long-horizon agent tasks. While excelling in tool invocation and error recovery, it requires dual M5 Ultra Mac Studios for local deployment and trails competitors like Zhipu and Kimi in coding benchmarks.
vLLM and Meta Superintelligence Labs provide Day 0 inference support for Muse Spark Open, an open-weight sparse MoE model with 816B total parameters and a 1M context window designed for long-horizon agentic coding. The release features native multimodal perception and optimized parallelism strategies across Nvidia, AMD, and TPU hardware, including speculative decoding via a trained DSpark draft model.
Reflection AI, founded by former Google DeepMind researchers, released Beam, its first open-weight model featuring a sparse Mixture-of-Experts (MoE) architecture with 501B total parameters but only 23B active per inference. Beam approaches the performance of Zhipu's GLM-5.2 on advanced reasoning benchmarks while requiring only 1/4 to 1/3 of the compute, targeting high-throughput coding and agent tasks. Concurrently, NVIDIA is in early-stage talks to invest in or acquire Reflection AI to strengthen its position in the open-model ecosystem.
According to the Financial Times, Nvidia is in early talks to acquire or further invest in Reflection AI, a $25B open-weight model startup. The deal may utilize an 'acqui-hire' structure—similar to Nvidia's previous Groq acquisition—to license technology and hire staff while bypassing antitrust scrutiny. This follows Reflection's recent release of Beam, a sparse MoE model claiming competitive performance with GLM 5.2 at significantly lower inference costs.