Microsoft Maia 200: The 3nm AI Chip Powering GPT-5.2 and Copilot Inside Azure

Every time you ask Copilot to draft an email or ChatGPT to write code, a chip somewhere has to turn that prompt into tokens — fast, and cheaply enough that the service can stay profitable. For years, that job belonged almost entirely to Nvidia GPUs. Microsoft now wants a bigger share of it for itself. Its answer is Maia 200, a custom AI chip built specifically for inference, and it is already running production workloads inside Azure.

Maia 200 is not a product you can buy. You will never see it in a store or rent one by the hour. But if you use Microsoft’s AI services, there is a good chance your requests are already passing through one.

What Is Microsoft Maia 200?

Maia 200 is Microsoft’s second-generation custom AI accelerator, announced on January 26, 2026. It follows Maia 100, the company’s first in-house AI chip, introduced in 2023 for large-scale Azure AI workloads. Where Maia 100 was a general-purpose first step, Maia 200 has a narrower, sharper job: inference — running already-trained models to generate text, answers, code and agent actions at massive scale.

That focus matters because inference, not training, is where most AI compute now goes. Training a frontier model is a rare, enormous event. Serving it to millions of users, millions of times a day, is the everyday cost — and the everyday bottleneck. Microsoft’s Scott Guthrie, executive vice president for Cloud + AI, described Maia 200 at launch as “a breakthrough inference accelerator engineered to dramatically improve the economics of AI token generation.” In other words, the chip exists to make each token cheaper.

Maia 200 Specifications at a Glance

The headline numbers, all published by Microsoft, put Maia 200 firmly in the top tier of AI accelerators:

  • Process: TSMC 3nm, with more than 140 billion transistors
  • Compute: over 10 PFLOPS at FP4 and over 5 PFLOPS at FP8
  • Memory: 216GB of HBM3e at 7 TB/s of bandwidth, plus 272MB of on-chip SRAM
  • Power: 750W thermal design power
  • Networking: 2.8 TB/s of bidirectional scale-up bandwidth per chip over standard Ethernet
  • Scale: clusters of up to 6,144 accelerators

The emphasis on FP4 and FP8 — very small number formats — is deliberate. Modern inference workloads increasingly run in low precision, which lets a chip do far more calculations per second on the same silicon. The large HBM3e memory pool matters just as much: big models need their weights close to the compute, and 216GB with 7 TB/s of bandwidth keeps the chip fed instead of waiting on data.

Close-up of an AI accelerator chip with glowing blue data pathways

Built for Inference, Not Training

Maia 200 is tuned for a world in which AI models are served, not trained. It is designed for high-volume token generation, agentic workflows that call models repeatedly, synthetic-data generation and reinforcement-learning workloads that lean heavily on inference.

That is a different design trade-off from a general-purpose training GPU. A training chip must handle huge gradient calculations and massive memory writes. An inference chip needs to answer quickly, handle many simultaneous requests, and sip power per token. Microsoft’s bet is that the second problem is now the bigger market — and that a purpose-built chip can solve it more cheaply than a general-purpose one.

Microsoft also ships a preview Maia SDK, the software layer developers use to optimize supported models for the chip. The SDK does not turn Maia into something customers can buy or run on their own hardware — it is a tool for tuning workloads that run on Microsoft’s own infrastructure.

Ethernet Networking: Clusters of Up to 6,144 Chips

One of Maia 200’s most unusual design choices is its networking. Instead of a proprietary interconnect fabric, Maia 200 links chips over standard Ethernet: 2.8 TB/s of scale-up bandwidth per chip, four chips per tray joined by direct links, and a two-tier network supporting clusters of up to 6,144 chips.

Using Ethernet is a bet on openness and scale. Proprietary fabrics can be faster in narrow cases, but Ethernet is a commodity technology with a huge ecosystem, familiar tooling and broad vendor support. For a company building data centers at Azure’s scale, that familiarity is worth a great deal — and it means Maia clusters can grow with standard networking gear rather than custom hardware at every step.

Where Maia 200 Is Running Today

Maia 200 is not a paper launch. Microsoft says the chip was in production by June 2026 in the Azure US Central region near Des Moines, Iowa, with US West 3 near Phoenix, Arizona next. Italy, Australia and South Korea have been named as the following regions.

The workloads already running on it are the ones Microsoft cares about most: OpenAI’s GPT-5.2 and Microsoft 365 Copilot. That pairing is no accident. GPT-5.2 is the flagship model behind many of Microsoft’s AI features, and Copilot is the product that puts AI in front of hundreds of millions of Office users. Running both on in-house silicon gives Microsoft direct control over the cost of its two most important AI services.

There is also a telling detail in what Microsoft has not announced: there is no Maia 200 virtual machine on Azure and no published price for the chip. Unlike Nvidia’s GPUs, Maia is infrastructure Microsoft keeps for itself. Press reports in May 2026 suggested Anthropic was in early talks to run workloads on Maia — which would make it the first outside company to do so — but neither company has confirmed a deal.

Rows of server racks inside a hyperscale Azure data center

How Maia 200 Stacks Up Against Trainium and TPU

Microsoft’s published comparisons are aggressive. The company says Maia 200 delivers three times the FP4 performance of Amazon’s third-generation Trainium chip, and FP8 performance above Google’s seventh-generation TPU. It also claims 30% better performance per dollar than the latest-generation hardware in its own fleet, and 40% better performance per watt when running its MAI models — though it has not stated what baseline that wattage figure is measured against.

These numbers deserve a clear caveat: they are Microsoft’s own comparisons, drawn against competitors’ published figures, not independent third-party benchmarks. Peak petaflops also do not translate directly into tokens per second or lower prices for users. The real test will be measured performance on production workloads over time — and whether the cost savings show up in cheaper AI services or simply in Microsoft’s margins.

Even so, the direction is unmistakable. Every major cloud provider is now building its own AI silicon: Amazon has Trainium, Google has its TPU line, and Microsoft has Maia. The hyperscalers are no longer just Nvidia’s biggest customers — they are increasingly its competitors. That shift is already visible elsewhere in the chip market, from AMD’s MI355X accelerators powering new Claude models to startups like SiMa.ai building AI chips built to run many models on one device.

What Maia 200 Means for Azure Users

For most people, Maia 200 will be invisible — and that is the point. You will not choose it in an Azure portal or see it on an invoice. What you may notice, over time, is faster responses from Copilot, more capable AI features in Microsoft 365, and new models arriving sooner because the infrastructure to serve them is cheaper to run.

For developers, the impact is indirect but real. Cheaper inference means Microsoft can afford to run more powerful models behind its APIs, including the models that power tools like Microsoft’s MAI Code 2 coding model. It also means the economics of AI agents — which burn through tokens with every step they take — become more sustainable. And as models keep getting bigger, from OpenAI’s GPT-6.2 Astra and Sol models to whatever comes next, the chip underneath them matters more, not less.

The honest version of the story is that Maia 200 is a cost and control play, not a revolution you will feel tomorrow. Microsoft is reducing its dependence on a single supplier, cutting the cost of its most expensive workloads, and building the infrastructure for an era in which AI inference is a utility. The chip itself is impressive — 3nm silicon, 140 billion transistors, 216GB of memory. The strategy behind it is bigger: own the silicon, own the economics, own the future of AI services.

The Bigger Picture: Every Hyperscaler Is Now a Chipmaker

Maia 200 is the clearest sign yet that the cloud AI market has entered its custom-silicon era. Training grabs the headlines, but inference pays the bills — and the companies that serve the most AI traffic now want to control the hardware that serves it. Microsoft’s second-generation chip, running real production workloads within months of its announcement, shows how fast that shift is happening.

The next question is whether Maia 200 stays a Microsoft-only advantage or opens up. If Anthropic or other partners do start running workloads on Maia, Microsoft’s chip becomes a platform, not just an internal cost saver. Until then, Maia 200’s biggest impact is the one users never see: cheaper tokens, faster Copilot answers, and an Azure that depends a little less on anyone else’s silicon. For more on the chip and its place in Azure’s AI strategy, see the complete Maia 200 guide, the Maia 200 specs and Azure access overview, and the Gadgets 360 report on the Maia 200 introduction.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top