November 16, 2023: Microsoft has unveiled a groundbreaking step in AI infrastructure by introducing two bespoke chips: the Azure Maia AI Accelerator and the Azure Cobalt CPU.
Located in a secretive lab on the Redmond campus, engineers have meticulously developed these chips, tailoring them to meet the rising demand for AI applications.
Azure Maia and Cobalt
The Azure Maia AI Accelerator, designed specifically for AI and generative AI tasks, and the Azure Cobalt CPU, an Arm-based processor, represent the culmination of Microsoft’s comprehensive approach to infrastructure systems.
This strategy encompasses everything from silicon selection, software, servers, to racks and cooling systems, ensuring that each element is optimized for Microsoft cloud and AI workloads.
Slated for early deployment next year in Microsoft’s datacenters, these chips will initially power Microsoft’s internal services like Microsoft Copilot and Azure OpenAI Service. They will be integrated into a broad spectrum of products from industry partners, addressing the urgent need for efficient, scalable, and sustainable computing power.
Microsoft’s venture into homegrown chips allows for a nuanced control over every infrastructure element, ensuring optimal performance for Microsoft cloud and AI workloads. The hardware will synergize with software, co-designed to unlock new capabilities and opportunities, aiming for a flexible and power-optimized Azure hardware system.
In addition to its custom silicon efforts, Microsoft is broadening its industry partnerships, introducing new infrastructure options for customers.
This includes collaborations with NVIDIA for the NC H100 v5 Virtual Machine Series and the upcoming NVIDIA H200 Tensor Core GPU, as well as AMD for the ND MI300 virtual machines.
The Future of Microsoft’s Infrastructure
This initiative marks a significant milestone in Microsoft’s journey to refine its technological stack, from core silicon to the end service.
By innovating deep within the stack, Microsoft ensures the future of customer workloads on Azure, focusing on performance, efficiency, and cost.
