Qualcomm argues that on-device AI is moving from single-purpose features such as imaging and speech transcription toward agentic experiences capable of intent understanding and task orchestration. Its approach pairs an 'on-device first' cloud-edge continuum with Mixture-of-Experts sparsity to cut per-inference compute, backed by NPU hardware and a full-stack software ecosystem.
Mixture-of-Experts (MoE)
混合专家
An architecture that sparsely activates multiple sub-networks to enhance capacity and efficiency.
Curated Intelligence
The open-source llama-modchip v2.4.0 claims it can run a 284B-parameter DeepSeek-V4-Flash MoE model entirely in RAM on a 64GB CPU-only workstation, peaking at 57.61 GiB with roughly 9-15 tok/s decode at 256K context. It relies on quantization and memory optimizations such as SwiGLU-H2, plus Fractal-64/MXFP4 transcoding and status tooling.
An independent researcher released MAPLE, a structural pruning method for mixture-of-experts LLMs that removes dormant experts, shrinking Gemma 4 26B-A4B from 9.92 GB to 4.75 GB (about 1.48 bits per parameter) so it runs entirely in 6 GB of laptop VRAM at 45.4 tokens per second. The checkpoint is specialized for Python coding and STEM reasoning, and the pipeline is claimed to be domain-agnostic.
AI chip design firm Ricursive Intelligence has released Ripple-65B Preview, an Apache 2.0 open model pretrained on 12 trillion tokens using a dense 65B Transformer rather than a mixture-of-experts design. It scores 80.52 on TerminalBench 2.1, the best among open US models, though it trails the newly released Inkling 2 Small on other benchmarks; Ricursive puts total R&D and training cost at just $32 million.
Microsoft declined to renew a 2023 Oracle contract built around Bing and is switching off the rented AI servers, because Oracle's hardware ran older-generation Nvidia chips that no longer matched Microsoft's specs. Under per-GPU-hour pricing, a Blackwell GPU delivers several times the inference output of an H100, so Hopper and Ampere hours now buy far less work. The move highlights how frontier mixture-of-experts models, with heavy memory and interconnect demands, are pushing AI capacity procurement from single-GPU performance toward rack-scale systems.
Moore Threads announced Day-0 adaptation of Moonshot's open-weight Kimi K3.1 Pro on its MTT S5000 AI card and MUSA software stack, covering model loading, service startup, and text/image inference. Key work includes adapting sparse attention DSA indexing and KV cache layout, online FP8 conversion of MXFP4 expert weights, and integrating visual attention. It signals faster domestic GPU software-stack readiness for trillion-parameter MoE models.
Thinking Machines, the startup founded by former OpenAI CTO Mira Murati, has released Inkling 2 Small, a 594-billion-parameter mixture-of-experts model that activates only 17 billion parameters per inference. It scores 31 on the Artificial Analysis Intelligence Index, an 18-point gain over its predecessor and roughly on par with Gemini 3.8 Flash, with notable improvements on agentic tasks, though cost per task is six times higher. The model debuts as a proprietary offering on the Tinker platform, with Apache-licensed weights to follow.
Moffett AI closed a second funding round of nearly RMB 1 billion within five months, backing its dual weight-and-activation sparsity algorithm and dedicated inference chip line. The piece argues that MoE-style conditional computation, exemplified by a 1.6-trillion-parameter model activating only ~49 billion parameters per token, is pushing sparse computing to the center of AI chip design where general-purpose GPUs cannot deeply co-optimize for sparsity patterns.
Thinking Machines Lab released Inkling 2 Small, a 594B-parameter Mixture-of-Experts model that activates only 17B parameters per token, routing each token to 8 of 384 experts across 48 layers with a 292M vision encoder for image input. It scores 31 on the Artificial Analysis Intelligence Index and 86.3% on SWE-Bench Pro v2, but is slow per task and weak on Terminal-Bench at 13.1%; weights remain closed with an Apache 2.0 release planned after safety testing.
Thinking Machines Lab, founded by former OpenAI CTO Mira Murati, is raising $5 billion at a $40 billion pre-money valuation after an earlier attempt at a $50 billion valuation fell through. The same day it released Inkling 2 Small, a 594B-A17B MoE model, with open weights under Apache 2.0 planned for next week.