SageMaker HyperPod Adds Deep Health Checks for Slurm Clusters
Amazon SageMaker HyperPod now supports deep health checks for Slurm-orchestrated clusters using continuous provisioning. This feature allows proactive verification of GPU accelerator health on running instances, preventing wasted compute time and delayed workloads due to unhealthy nodes. It enables targeting specific instances or groups for stress and connectivity tests before committing resources, with results visible via SageMaker console and APIs. This enhancement is available in all regions where SageMaker HyperPod is offered.
- →Deep health checks for Slurm clusters with continuous provisioning
- →Automatic instance isolation and recovery for unhealthy nodes
- →Availability and further information
Features (1) ›
- Deep health checks for Slurm clusters with continuous provisioning
SageMaker HyperPod now includes deep health checks for Slurm-orchestrated clusters using continuous provisioning. This allows proactive verification of GPU accelerator health on running instances at any time, ensuring hardware is validated before jobs are scheduled. Progress and results are visible through the SageMaker console and APIs.
Enhancements (1) ›
- Automatic instance isolation and recovery for unhealthy nodes
Instances undergoing health checks are automatically isolated from workload scheduling and returned to service upon passing. Failed instances are automatically rebooted or replaced when paired with HyperPod's automatic node recovery capability, ensuring overall cluster health.
Notes (1) ›
- Availability and further information
This new capability is available in all regions where Amazon SageMaker HyperPod is offered. More details can be found in the Amazon SageMaker HyperPod User Guide.
https://aws.amazon.com/about-aws/whats-new/2026/07/deep-health-check-continuous-slurm/