Rohan Paul (@rohanpaul_ai) · 10/12/2026, 21:52:07
Microsoft Announces Maia 300 Chips to Replace NVIDIA for Primary Inference: Multi-Chiplet Architecture Boosts Tokens per Megawatt by 40%
Microsoft CEO Satya Nadella officially announced that the in-house Maia 300 chips will become the primary inference infrastructure for Microsoft products starting in 2027, significantly reducing reliance on NVIDIA and AMD hardware. Featuring a multi-chiplet design optimized for long-context reasoning and low-latency agent tasks, Maia 300 claims to deliver 40% more tokens per megawatt than leading GPU-based systems. By addressing data movement bottlenecks rather than just raw compute, the new architecture achieves up to 7x higher token throughput compared to its predecessor, Maia 200.
SOURCE COVERAGEFull Original Report
Rohan Paul (@rohanpaul_ai) · 2026-10-12 13
Satya Nadella officially announced that Microsoft will make its in-house Maia chips the primary inference system for its products, scaling to multiple gigawatts.
This will make it less reliant on NVIDIA hardware, and Maia 300 is meant to replace AMD chips too, not only Nvidia. The key claim is 40% more tokens per megawatt than leading GPU-based systems
A full rack packs 96 accelerators into 167kW with 27.6TB of HBM, and Microsoft claims it delivers up to 7x the token throughput of a Maia 200 rack.
The shift will start in 2027 with Maia 300, a third-generation accelerator built for long-context reasoning, low-latency chat and multi-step agent tasks.
Its predecessor, Maia 200, already serves production traffic including Microsoft Copilot and Microsoft claims Maia 200 produces up to 1.7x more tokens per watt than leading GPU infrastructure.
Maia 300 goes after the cost of moving data, which Microsoft says now limits large inference systems more than raw arithmetic does.
Maia 300 uses a “multi-chiplet” design, A traditional chip is one large slab of silicon that does everything: the math (compute), managing the memory (memory controller) and talking to other chips (I/O). Maia 300 instead splits those jobs into several smaller pieces of silicon, called chiplets, packaged together so they act as one chip. That leaves more room on the compute chiplets for the parts that actually run the AI math, and each chiplet can be made with whatever manufacturing process suits its job best.
Each chip pairs 8 HBM4 stacks delivering 18.0 TB/s of peak bandwidth with a 950W TDP, and uses on-die SRAM as a scratchpad to relieve that memory.
Support for low-precision formats such as an enhanced MXFP4 shrinks the KV cache, the stored attention state that grows with every token, so longer contexts fit within the same power budget.
Supply is the harder constraint, since Microsoft has reportedly been negotiating with TSMC for more than 300,000 Maia 300 units for 2027 delivery, a figure still under negotiation rather than a signed purchase order.
↳ 引用 @satyanadella (Satya Nadella): Today we are announcing our biggest bet yet on our own silicon.
Starting next calendar year, we will scale our Maia family to multi-gigawatt deployments, making it the primary inference system for our own product and services.
And we’re introducing Maia 300, which delivers 40% more tokens per watt compared to leading GPU-based systems.
We are already deploying Maia 200 across the US, Europe, and Australia, delivering up to 1.7x more tokens per watt across experiences in Teams and Copilot.
Our goal is to build on the full-stack efficiency gains we’ve already seen, while continuing to bring the latest innovations from across the ecosystem into our fleet.
