Amazon EMR on EKS now supports Spark Connect for interactive workloads
Amazon EMR on EKS now supports interactive Apache Spark sessions via Spark Connect, enabling data engineers and data scientists to develop and debug applications. This allows interactive development from managed notebooks in Amazon SageMaker Unified Studio or popular IDEs like Jupyter and Visual Studio Code, leveraging a persistent Spark context. The client-server architecture decouples the application client from the Spark driver, maintaining preferred development environments while Spark runs on EKS clusters. This feature is available with EMR release 7.14 (Apache Spark 3.5) and emr-spark-8.1.0 (Apache Spark 4.1) in all AWS Commercial Regions.
Features (1) ›
- Interactive Spark Sessions with Spark Connect on EMR on EKS
Amazon EMR on EKS now provides interactive Apache Spark sessions using Spark Connect. This enables data engineers and data scientists to develop and debug applications from SageMaker Unified Studio or popular IDEs, maintaining a persistent Spark context. The client-server architecture decouples the client from the Spark driver, supporting flexible development workflows.
https://aws.amazon.com/about-aws/whats-new/2026/09/emr-eks-spark-connect-interactive/
Related releases
- AWS Named a Leader in Gartner's 2026 Magic Quadrant for Container Management AWS Containers Blog ·
- Standardizing Edge Container Deployments with Amazon EKS: A Strategy Guide AWS Containers Blog ·
- Amazon SageMaker HyperPod Inference Gateway for Scalable LLM Inference AWS What's New ·
- Amazon EMR on EKS Now Supports IPv6 for Scalable Big Data Workloads AWS What's New ·
- Building a single-pane NOC dashboard for Amazon EKS with Amazon CloudWatch AWS Containers Blog ·
- Amazon EMR Introduces Long Term Support for Apache Spark with Version 4.1 AWS What's New ·