Amazon EMR on EC2 adds interactive Spark Connect support
Amazon EMR on EC2 now supports interactive Apache Spark sessions using Spark Connect. This enables data engineers and scientists to develop and debug Spark applications interactively from notebooks and IDEs connected to dedicated EMR on EC2 clusters. The feature decouples the client from the Spark driver, allowing flexible development workflows and providing enhanced observability. It is available on EMR on EC2 with AWS runtime for Apache Spark 8.0 and later in most AWS Regions.
- →Enable interactive Spark development with Spark Connect on EMR on EC2
- →Decouple Spark client from driver for flexible development
- →Enhance observability for interactive Spark sessions
- →Availability and access details for interactive Spark Connect
Features (1) ›
- Enable interactive Spark development with Spark Connect on EMR on EC2
Amazon EMR on EC2 now supports Spark Connect, allowing interactive Apache Spark sessions from managed notebooks and IDEs. This feature enables data engineers and scientists to develop and debug Spark applications remotely on dedicated EMR on EC2 clusters.
Enhancements (2) ›
- Decouple Spark client from driver for flexible development
Spark Connect's client-server architecture separates the application client from the Spark driver. This allows users to maintain their preferred development environment and tooling while Spark infrastructure runs on the cluster, supporting ad hoc exploration and iterative debugging.
- Enhance observability for interactive Spark sessions
Users can monitor and debug active and completed sessions through the EMR console, Spark UI, and Spark History Server. Session management is also available via EMR console or API/CLI/SDK.
Notes (1) ›
- Availability and access details for interactive Spark Connect
Interactive Sessions are available on Amazon EMR on EC2 with AWS runtime for Apache Spark 8.0 and later. The SageMaker Unified Studio experience is also available in supported regions, excluding AWS GovCloud and China Regions.
https://aws.amazon.com/about-aws/whats-new/2026/08/amazon-emr-ec2-spark-connect/
Related releases
- Amazon EC2 R8a Instances Now Available in Canada (Central) Region AWS What's New ·
- AWS EC2 Image Builder streamlines Microsoft Office integration Modernizing with AWS Blog ·
- AWS EC2 High Memory U7in Instances Now Available in São Paulo AWS What's New ·
- Amazon EC2 introduces application status checks for app-level monitoring AWS What's New ·
- Amazon EC2 R8i and R8i-Flex instances available in Europe (Milan) AWS What's New ·
- Amazon GameLift Servers adds 21 EC2 instance types AWS What's New ·