Azure Maia 300: Microsoft details its new in-house chip for AI agent inference

On October 12, 2026, Microsoft published the architecture of Maia 300, its new in-house AI accelerator, on its Azure blog. The post is signed by Rani Borkar, president of Azure hardware systems and infrastructure, and by Saurabh Dighe. The chip is designed for inference, meaning running models rather than training them. It specifically targets AI agents, which chain steps together, call tools and work over long contexts. Microsoft cites Copilot, GitHub Copilot and its Foundry platform as use cases.

What Microsoft is announcing

• Multi-chip design (chiplets): the compute die is manufactured by TSMC on its N2P process. Memory and input/output are placed on separate chips.
• Memory: eight HBM4 memory stacks per accelerator, for 288 GB and a bandwidth of 18 TB/s.
• Rack: 96 accelerators spread across 24 trays, twice as many as the previous generation. A rack draws 167 kW, and liquid cooling removes more than 80% of its heat.
• Cluster: up to 12,288 accelerators spread across 128 racks.

All the comparisons in Microsoft's post are against its previous chip, Maia 200. The company claims up to 7 times more token throughput at the scale of a full rack and more than 2.5 times more performance per watt. It also claims 2.5 times more memory bandwidth and 33% more memory capacity. According to LeMagIT, each chip draws 950 W, compared with 750 W for Maia 200.

Versus Nvidia and AMD

Microsoft's post does not compare Maia 300 with Nvidia or AMD graphics processing units (GPUs), although SiliconANGLE reports a Microsoft estimate of 40% more tokens per megawatt than "leading GPU-based systems." LeMagIT ran the comparison on memory. Nvidia's Rubin R100 offers 288 GB at 22 TB/s and AMD's MI455X 432 GB at 23.3 TB/s, versus 288 GB at 18 TB/s for Maia 300. The outlet estimates that Microsoft's chip is ahead of the other hyperscalers' chips on paper, but remains behind Nvidia and AMD GPUs.

Like Google with its TPUs and AWS with Trainium, Microsoft is seeking to depend less on GPU suppliers, whose supply remains tight. According to LeMagIT, the chip is due to be available in 2027. Microsoft promises to say more at its Ignite conference in November.

Key takeaways

• No independent measurement: all the announced gains come from Microsoft and, in its post, are valid only relative to its own previous chip.
• No direct effect on customers: Maia 300 is not for sale. Businesses will only see its effects through Azure, Copilot and Foundry services.
• Pricing: according to LeMagIT, Microsoft has not announced any price cuts for its AI services.
• Location: Microsoft does not say in which regions, in Europe or elsewhere, Maia 300 will be deployed.