Break-Glass Access for Amazon EKS When Federated Identity Fails
AWS has published a detailed pattern for implementing break-glass access for Amazon EKS clusters, designed to prevent lockouts during federated identity provider outages. This guidance outlines a dedicated, cross-account emergency path that relies solely on AWS IAM and STS, removing circular dependencies inherent in primary federated authentication. It leverages pre-provisioned roles with MFA and EKS Cluster Access Management, ensuring administrators can restore access without external identity systems. The pattern includes Infrastructure-as-Code templates, a validation method, and a post-incident procedure for EKS cluster operators and architects.
- →The Problem: EKS Lockout During Federated Identity Outages
- →Limitations of Existing Break-Glass Recommendations
- →Solution Overview: A Dedicated Emergency Path
- →Authentication Chain and Pre-Provisioning
- →Key Prerequisites for Implementation
Notes (5) ›
- The Problem: EKS Lockout During Federated Identity Outages
Production Amazon EKS clusters typically authenticate users via federated identity providers, but outages or misconfigurations can lock administrators out. This creates a circular dependency where cluster access is needed to fix the identity provider, but the provider is required for access.
- Limitations of Existing Break-Glass Recommendations
While AWS provides guidance on emergency users and EKS misconfiguration recovery, a worked pattern for the Kubernetes authorization layer was missing. Existing recommendations often rely on the cluster creator role, which may lack MFA or become invalid during organizational changes.
- Solution Overview: A Dedicated Emergency Path
The presented pattern ensures the emergency path shares no dependencies with the primary path, using AWS IAM and STS for authentication. It employs a dedicated cross-account role with MFA and manages authorization via the Amazon EKS Cluster Access Management (CAM) API, moving control out of the cluster.
- Authentication Chain and Pre-Provisioning
The pattern involves pre-provisioning the emergency role, access entry, and policy association, leaving them dormant until needed. The authentication chain, from AWS IAM to AWS STS to the EKS authentication webhook, is independent of any external identity provider, ensuring resilience during outages.
- Key Prerequisites for Implementation
Before deployment, the EKS cluster authentication mode must be 'API' or 'API_AND_CONFIG_MAP', and a separate operations AWS account with MFA-registered IAM principals must exist. Crucially, network reachability to the EKS API endpoint must also be separately ensured, as this pattern only restores authentication and authorization.
https://aws.amazon.com/blogs/containers/break-glass-access-for-amazon-eks-when-federated-identity-fails/
Related releases
- GPU-accelerated Apache Spark on Amazon EMR with EC2 G7 instances runs up to 3.7x faster AWS Big Data Blog ·
- Amazon EKS now supports multiple external OIDC identity providers per cluster AWS What's New ·
- SageMaker HyperPod Enhances Ray Support with Observability, Resilient Training, and Accelerated Inference AWS What's New ·
- Amazon EKS Capability for Argo CD adds custom configuration support AWS What's New ·
- AWS Local Zone in Las Vegas, Nevada, Now Generally Available AWS What's New ·
- Amazon EKS now supports certificate authority (CA) rotation with automated management AWS What's New ·