ReleaseBytes
aws AWS Big Data Blog ·

Amazon EMR Advanced Managed Scaling Offers Cost-Performance Tuning

blogdataawsengineerhealthcareaws-ec2
feature

Amazon EMR on EC2 now offers Advanced Managed Scaling, allowing users to explicitly define their cluster scaling priorities between cost efficiency and job performance. Previously, customers relied on default EMR Managed Scaling or custom autoscaling rules, each with limitations. This new capability provides a utilization-performance slider to tailor scaling strategies, impacting everything from how aggressively nodes are added or removed to how quickly jobs complete, enabling better optimization for diverse workload needs.

  • →Advanced Managed Scaling for EMR on EC2
  • →New Scaling Strategies and Controls
  • →Real-time Workload Evaluation and Strategic Scaling
  • →API Access and Configuration Examples
Features (1) ›
  • Advanced Managed Scaling for EMR on EC2

    Amazon EMR on EC2 has introduced Advanced Scaling, building on EMR Managed Scaling to provide granular control over cluster scaling behavior. Users can now set a UtilizationPerformanceIndex (values 1, 25, 50, 75, 100) to prioritize cost savings with conservative scaling, balanced performance and utilization, or aggressive scaling for performance-sensitive workloads.

Notes (3) ›
  • New Scaling Strategies and Controls

    The UtilizationPerformanceIndex maps to specific strategies: 'Utilization optimized' (1) prioritizes cost by scaling up less aggressively and releasing nodes slowly; 'Balanced' (50) offers a mix of price-performance for steady or mixed workloads; and 'Performance optimized' (100) prioritizes rapid job completion for SLA-sensitive jobs with aggressive scaling.

  • Real-time Workload Evaluation and Strategic Scaling

    Advanced Scaling continuously evaluates workloads by factoring in pending tasks, memory pressure, and executor demand to adjust cluster size. This enables strategic scaling policies throughout the day, aligning resource allocation with specific operational patterns and business needs.

  • API Access and Configuration Examples

    The Advanced Scaling capability is available via API, allowing users to modify existing cluster configurations by setting the UtilizationPerformanceIndex. The blog post includes examples demonstrating the performance and scaling behavior for utilization-optimized, balanced, and performance-optimized configurations using a TPC-DS dataset.

Read the original announcement →

https://aws.amazon.com/blogs/big-data/optimizing-costs-and-performance-with-advanced-managed-scaling-on-amazon-emr-on-ec2/

Related releases