Microsoft Launches Maia 200 in US Azure Data Centers
Microsoft has brought its Maia 200 AI inference accelerator to the US Central Azure region. This is a big step forward for the company’s infrastructure. The company says that Maia 200 is its most advanced inference-focused chip, made just for big cloud workloads.
The release shows how Microsoft is working harder to combine silicon, networking, and software in its Azure cloud ecosystem. The next area where Maia 200 will be deployed is US West 3 in Arizona.
Source: Tom’s Hardware/Website
Chip Architecture Focuses on Inference Performance Efficiency
The Maia 200 is made with TSMC’s 3-nanometer process and has built-in FP8 and FP4 tensor cores. Its design is aimed at inference workloads by finding the right balance between compute density, memory throughput, and energy efficiency at scale.
The accelerator has HBM3e memory built in, which can send up to 7 terabytes of data per second. Extra memory and data movement engines on the chip help keep big AI models running at full speed.
Microsoft Claims Major Gains Over Rival AI Accelerators
Microsoft says that Maia 200 is 3 times faster at FP4 inference than Amazon’s 3rd-generation Trainium accelerator. The company also says that FP8 performance is better than that of Google’s 7th-generation tensor processing unit.
Microsoft says that these improvements mean that Azure inference systems are about 30% more cost-effective than they are now. The company hasn’t confirmed yet whether the product will be available outside of the US.
Recommended Article: China Prepares H200 AI Chip Purchases as Nvidia Visits Shanghai
Custom Rack Design Enables Dense Inference Clusters
Using trays with 4 chips per unit, Maia 200 accelerators are set up in racks. To keep bandwidth and lower communication latency, each tray connects directly to the other trays without switching.
The Maia AI transport layer uses the same protocol for communication within a rack and between racks. This method lets clusters grow without adding too many network hops or wasted capacity.
2 Tier Scale Up Model Built on Standard Ethernet
Microsoft built Maia 200 on a 2 tier scale-up architecture that uses standard Ethernet instead of proprietary fabrics. A tightly integrated network interface card makes things more reliable, predictable, and cheaper to run.
Each accelerator can handle up to 1.4 terabytes per second of dedicated scale-up bandwidth. This lets clusters with more than 6,000 accelerators work together.
Lower Power Usage and Total Ownership Costs Targeted
Microsoft says that Maia 200 makes power use more efficient while keeping the same inference throughput across dense deployments. These improvements help lower the total cost of owning Azure data center operations.
The architecture is made to work with Microsoft’s cloud fleet all over the world without losing predictability. This means that enterprise customers who run AI inference workloads that are sensitive to latency can use Maia 200.
Software Co Development Shaped Maia 200 Design
Microsoft used a complex simulation pipeline to model how large language models would work and talk to each other in the past. This let engineers improve silicon, networking, and system software all at once.
A wide range of emulation environments made it possible to test everything from the kernel level to the full model execution. A preview release of the Maia 200 software development kit will give developers early access.
