AWS ARC Zonal Shift integrates with EKS Auto Mode and Karpenter
This post details how Amazon Application Recovery Controller (ARC) Zonal Shift integrates with Amazon Elastic Kubernetes Service (EKS) Auto Mode and Karpenter to automatically recover workloads from Availability Zone failures. When a zonal impairment occurs, the integration cordons nodes, removes pod IPs from load balancers, and temporarily suspends new capacity provisioning in the affected zone. It also provides guidance on best practices for application resilience and how to enable Zonal Shift on both self-managed Karpenter and EKS Auto Mode clusters.
- →Zonal Shift integration with EKS Auto Mode and Karpenter overview
- →Karpenter's detection and recovery from zonal impairments
- →Best practices for application resilience
- →AWS Resilience Hub for identifying exposure
- →Enabling Zonal Shift for OSS Karpenter
Enhancements (3) ›
- Karpenter's detection and recovery from zonal impairments
Karpenter now detects Zonal Shift status by polling the ARC GetManagedResources API every 30 seconds. It resumes provisioning capacity in all configured zones once service to impaired zones is restored or a zonal shift expires. Nodes in affected zones are tainted, preventing pod scheduling, and Karpenter avoids disrupting these nodes until the shift expires.
- Enabling Zonal Shift for OSS Karpenter
Enabling Zonal Shift for self-managed Karpenter requires Karpenter version 1.12.0 or higher, setting the --enableZonalShift flag to true on the Karpenter controller, and granting the Karpenter controller's IAM role permission to call the ARC GetManagedResources API. Zonal Shift must also be enabled on the EKS cluster. Terraform examples and scripts are available to assist with installation and configuration.
- Enabling Zonal Shift for EKS Auto Mode
Enabling Zonal Shift on EKS Auto Mode clusters is a single step, as the mode detects shift signals internally without requiring changes to the cluster's IAM configuration, Helm flags, or service-linked roles. Enabling Zonal Shift on the cluster makes it eligible for manual shifts and registers it as a managed resource in ARC. For automatic shifts during zonal impairments, Zonal Autoshift must be separately enabled through the ARC console or API.
Notes (4) ›
- Zonal Shift integration with EKS Auto Mode and Karpenter overview
Amazon ARC Zonal Shift features are now supported by the AWS provider for Karpenter, enabling automated traffic and capacity shifting away from failed Availability Zones (AZs). This integration automatically cordons nodes in affected zones, deregisters pod IPs from load balancers, and prevents new capacity provisioning in impaired zones. It also suspends voluntary disruptions for EKS Auto Mode and self-managed Karpenter during zonal impairments.
- Best practices for application resilience
To improve application reliability during zonal failures, it's recommended to distribute pods across Availability Zones using topology spread constraints or node affinity, avoid single-replica deployments, configure pod disruption budgets (PDBs), and manage stateful services with zonal resilience in mind, including EBS volume snapshots and replication.
- AWS Resilience Hub for identifying exposure
The next generation of AWS Resilience Hub uses generative AI to assess failure modes against AWS Well-Architected best practices, identifying issues like singleton deployments and missing multi-AZ spread. It maps findings to resilience policies and provides guidance for resolution, helping to determine which workloads warrant Zonal Shift.
- Testing Zonal Shift with a live EKS Auto Mode cluster
A full end-to-end test cycle is provided for EKS Auto Mode clusters, involving deploying a multi-AZ workload with topology spread constraints, verifying the pre-shift state, and triggering a manual Zonal Shift using the ARC API with a specified expiry time.
https://aws.amazon.com/blogs/containers/arc-zonal-shift-support-for-eks-auto-mode-and-karpenter/
Related releases
- Amazon EKS now supports advanced Kubernetes control plane configuration AWS What's New ·
- Amazon EKS 1.34+ enables forensic container checkpointing with Kubelet API AWS Containers Blog ·
- Amazon EKS introduces advanced Kubernetes control plane configuration AWS Containers Blog ·
- Amazon EKS Auto Mode automates node failure detection and repair AWS Containers Blog ·
- AWS Network Firewall adds forward proxy functionality AWS What's New ·
- AWS EMR on EKS Adds Apache Spark Troubleshooting Agent AWS Big Data Blog ·