Alibaba released Qwen4-Max, a 2.4T-parameter sparse MoE flagship that activates only 71B parameters per token at $1.5 per million input tokens. Its rebuilt sparse MoE stack uses three Gated DeltaNet layers to compress history for each Qwen Sparse Attention layer, a lightweight indexer for context selection, plus four gated residual branches and 328B N-gram embedding parameters. It tops 4 of 16 benchmarks but trails Claude Opus 5.5 and Kimi K3.1 on coding and terminal tasks, a clear trade-off.
Gated DeltaNet
门控 Delta 网络
An efficient sequence modeling method combining linear recurrence with gating mechanisms.
Curated Intelligence
Alibaba's Qwen team has officially launched Qwen4-Max, a 2.4-trillion-parameter sparse MoE flagship with native vision understanding and a 1M-token context window, claiming over 60% lower task cost at comparable performance. The model introduces systematic architectural changes including Gated DeltaNet, Qwen Sparse Attention, Gated Residual and N-gram Embedding, targeting long-horizon professional work in office, coding and research, and is now available via QwenCloud API with smaller open-source variants to follow.
Developer project qwenGPU compiles a 45-layer SIMT architecture directly into Qwen3.5-2B weights, enabling the model to execute real CUDA instructions internally. By repurposing Gated DeltaNet states as memory, it achieves high kernel fidelity on Tesla T4 hardware despite slower inference speeds and minor text drift.
Nace AI has open-sourced Drex Max v1, a decision model that outputs probability distributions over options via a single forward pass instead of generating text. Featuring a hybrid architecture of Gated DeltaNet and full-attention layers, it outperforms competitors on public decision benchmarks and supports efficient local deployment on hardware like Apple Silicon.
The nesformer project compiles the NES 6502 CPU, RAM, and ROM logic directly into standard Qwen3.5 model weights without training, enabling in-model simulation. By leveraging Gated DeltaNet states as machine memory, the LLM predicts CPU execution logs token-by-token, allowing classic games like Super Mario Bros. to run entirely within approximately 1GB of weight space.
Developer Divyansh Agrawal released Qwenputer, an experimental model based on Qwen3.5-4B that embeds a 32-bit RISC-V CPU into neural weights using Gated DeltaNet states as RAM. This enables native precise arithmetic and program execution without external tools, boosting large integer accuracy from ~50% to over 97%, though at the cost of doubling model size to 18.8GB.
Researcher Divyansh Agrawal introduced Qwenputer, an experimental model embedding a full RISC-V CPU directly into the weights of Qwen 3.5 4B. By utilizing Gated DeltaNet attention layers as RAM, the architecture enables the model to execute arithmetic operations and C/MicroPython code internally during forward passes, eliminating the need for external runtimes or tool calls.