SageMaker Inference Adds NVIDIA G7 Instances for Faster LLM Deployment
Amazon SageMaker now supports G7 instances powered by NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs for AI inference. These instances offer up to 4.6x performance improvement over previous generations, with higher GPU memory and networking, making them suitable for cost-effectively serving medium-to-large generative AI models. G7 instances are available for various inference workloads in select AWS regions.
- →SageMaker AI inference now supports G7 instances
- →G7 instances address cost and performance needs for generative AI
- →G7 instances are well-suited for specific model sizes and workloads
- →Deployment options and availability
Features (1) ›
- SageMaker AI inference now supports G7 instances
SageMaker AI inference now supports G7 instances powered by NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs. This update enables enhanced performance for deploying machine learning models, offering up to 4.6x AI inference performance compared to previous-generation G6 instances.
Notes (3) ›
- G7 instances address cost and performance needs for generative AI
G7 instances are designed to address the need for high GPU throughput and memory capacity for generative AI models. They provide 32 GB of GPU memory per GPU, 5th Generation Tensor Cores, up to 700 Gbps of EFA-enabled networking, and up to 7.6 TB of local NVMe SSD storage.
- G7 instances are well-suited for specific model sizes and workloads
These instances are optimized for serving models in the 7B–30B parameter range, image and video generation workloads, and multi-model inference endpoints. They benefit from higher memory bandwidth and throughput.
- Deployment options and availability
Models can be deployed on G7 instances via the SageMaker AI Inference console, API, or SDK by specifying G7 instance types in the endpoint configuration. G7 instances are available in US East (N. Virginia, Ohio) and US West (Oregon).
https://aws.amazon.com/about-aws/whats-new/2026/07/amazon-sagemaker-ai-g7/
Related releases
- AWS Glue Data Quality Adds Distribution Statistics for Data Profiling AWS What's New ·
- Amazon SageMaker AI inference adds G6 instances in AWS GovCloud AWS What's New ·
- Amazon SageMaker G7e instances expand to Seoul, London, and Tokyo AWS What's New ·
- SageMaker Studio Integrates with Amazon OpenSearch for Unified Data Analysis AWS What's New ·
- SageMaker Studio adds custom visual transforms for ETL AWS What's New ·
- Amazon RDS for SQL Server adds support for SQL Server 2025 AWS What's New ·