ReleaseBytes
azure Microsoft Azure Blog ·

AT&T and Microsoft leverage Foundry and AMD for trillion-token AI workloads

blogaiazureengineer
feature announcement

AT&T has developed its Open Telco (OTel) 2.0 AI models using Microsoft Foundry Managed Compute and AMD GPUs to handle trillion-token workloads. This partnership addresses the need for domain-specific AI in telecommunications by offering a scalable, cost-effective platform for training and deploying models. The solution enables flexibility in model choice and infrastructure, allowing organizations to accelerate AI development and production deployment.

  • →Foundry Managed Compute for scalable GPU capacity
  • →Multi open-model strategy for flexibility and cost control
  • →AT&T develops domain-specific AI with Microsoft Foundry and AMD
  • →Accelerated AI deployment speeds
  • →Cost optimization for large-scale AI
Features (2) ›
  • Foundry Managed Compute for scalable GPU capacity

    Microsoft Foundry Managed Compute provides streamlined access to dedicated GPU capacity, abstracting infrastructure management for AI development. AT&T utilized approximately 530 GPUs through this service, including 430 AMD Instinct™ MI300X GPUs, to experiment, optimize, and process large volumes of telecom data.

  • Multi open-model strategy for flexibility and cost control

    AT&T adopted a multi open-model strategy, deploying models like Phi-4, OSS-120B, and Gemma-4 from Hugging Face to support various stages of OTel2.0 development. Phi-4 processed over 700 billion tokens monthly for data preparation, demonstrating cost savings compared to frontier models.

Enhancements (2) ›
  • Accelerated AI deployment speeds

    Foundry Managed Compute allows for rapid deployment and scaling of models, reducing provisioning cycles. AT&T could deploy and scale models in days, accelerating development timelines for OTel2.0 compared to traditional weeks-long infrastructure availability.

  • Cost optimization for large-scale AI

    By using open models on Microsoft Foundry Managed Compute, AT&T achieved significant cost savings, processing approximately 1 trillion tokens for OTel2.0. This approach saved tens of millions of dollars compared to using frontier models, allowing for greater investment in experimentation.

Notes (1) ›
  • AT&T develops domain-specific AI with Microsoft Foundry and AMD

    AT&T has created its Open Telco (OTel) 2.0 AI models, designed for the telecommunications industry, using Microsoft Foundry Managed Compute and AMD Instinct™ MI300X GPUs. This platform allows for handling trillion-token workloads, offering flexibility in model choice and infrastructure optimization.

Read the original announcement →

https://azure.microsoft.com/en-us/blog/att-and-microsoft-scale-trillion-token-workloads-with-microsoft-foundry-and-amd/

Related releases