Amazon EMR on EKS adds Apache Spark troubleshooting agent
Amazon EMR on EKS now integrates an Apache Spark troubleshooting agent, enabling data engineers to diagnose job failures via natural language and receive automated root cause analysis. This feature, which analyzes Spark History Server data and logs, expands to cover all EMR deployment options and is accessible via the EMR on EKS console or compatible AI coding agents. It provides read-only, IAM-authenticated, and CloudTrail-logged diagnostics for issues like memory errors and data skew.
- →Apache Spark troubleshooting agent for EMR on EKS
- →Access troubleshooting agent via EMR console or AI agents
- →Troubleshooting agent now supports all EMR deployment options
- →Secure and auditable operations for the troubleshooting agent
Features (2) ›
- Apache Spark troubleshooting agent for EMR on EKS
The Apache Spark troubleshooting agent is now available for Amazon EMR on EKS, allowing data engineers to diagnose job failures using natural language and receive automated root cause analysis and code recommendations. This agent analyzes Spark History Server data, executor logs, and cluster configurations to identify common issues.
- Access troubleshooting agent via EMR console or AI agents
The agent can be accessed directly from the EMR on EKS console with a "Troubleshoot with AI" option for failed jobs. It is also available through the MCP (Model Context Protocol) using compatible AI coding agents such as Kiro, Claude Code, and Cursor.
Enhancements (1) ›
- Troubleshooting agent now supports all EMR deployment options
With this update, the Spark troubleshooting agent now covers EMR on EC2, EMR Serverless, and EMR on EKS, providing a unified experience for diagnosing job failures across all EMR deployment methods.
Notes (1) ›
- Secure and auditable operations for the troubleshooting agent
All operations performed by the troubleshooting agent are read-only, authenticated with IAM roles, and logged in AWS CloudTrail, ensuring security and auditability. The agent is available in AWS Regions where SageMaker Unified Studio is supported.
https://aws.amazon.com/about-aws/whats-new/2026/07/amazon-emr-eks-spark-troubleshooting/