Disable scale actions for AWS EMR clusters

Document created by natalia.ladiko@turbonomic.com Expert on Sep 10, 2019
Version 1Show Document
  • View in full screen mode

This document describes how to disable scale actions for AWS EMR in Turbonomic by creating a dynamic group filtered to appropriate tags. 

 

What is AWS EMR? 

AWS EMR is a big data platform which allows to process vast amounts of data quickly and cost-effectively at scale. AWS EMR leverages Amazon EC2 and Amazon S3 storage to create short-lived clusters which automatically scale to meet demand. Please see detailed description here - Amazon EMR - Amazon Web Services

 

Since AWS EMR clusters are ephemeral and automatically scale based on demand, trying to resize individual EC2 instances within an EMR cluster will result in cluster re-deployment with its initial size and configuration. To make Turbonomic aware of this behavior, resize recommendations should be disabled for AWS EMR clusters. 

 

Please follow the below steps to create a dynamic group based on tags

   1. In the Turbonomic instance, go to "Settings" -> "Groups" -> "Create New Group" -> "Virtual Machine"

   2. Select filter by "Tag" -> "Equals (RegEx)" -> and filter by the following tag criteria:

         .*aws:elasticmapreduce:instance-group-role.*

 

 

3. Click on "Save Group".

 

Next, please follow the below steps to create a policy to disable resize recommendations on the group that we created above:

   1. In the Turbonomic instance, go to "Settings" -> "Policies" -> "New Automation Policy" -> "Virtual Machine"

   2. Name the policy, in the scope section click on "Add Virtual Machine Groups" and search for the above created EMR group, in        this case "Natalia EMR Cluster Group"

   3. Then, expand "Action Automation" and select the following:

      - all "vCPU Resize ..." options and choose "Disable" on all

      - all "vMem Resize ..." options and choose "Disable" on all

   4. Click "Save and Apply"

 

All done.

 

A new policy to disable all resize recommendations called "Natalia EMR Cluster Group Disable Resize Policy" has been created for a group called "Natalia EMR Cluster Group". In this case, all Virtual Machines in the "Natalia EMR Cluster Group" group will not have any resize actions recommended by Turbonomic. 

Attachments

    Outcomes