Everything about AWS ECS(with hands-on)

This is fourth article of the series Everything about AWS ECS where I am going to show EC2 AutoScaling and Service AutoScaling in action. For the other articles in this series please click on following links:

1. ECS Overview and Task Definition
2. Cluster
3. Service
4. Load Testing
5. CI/CD Pipeline

We created 2 types of AutoScaling:

1. EC2 AutoScaling : This will increase or decrease the number of EC2 Instances based on the metric CapacityProviderReservation.
2. Service AutoScaling : This will increase or decrease the number of tasks based on the metric ECSServiceAverageCPUUtilization.

To summarize what we are going to do is, we will increase the traffic on our nginx server using a tool which will increase the value of ECSServiceAverageCPUUtilization to more than 70. This will trigger the Service AutoScaling alarm which will increase the value of Desired Task by 1 and then provisions a new task.

Now EC2 AutoScaling comes into picture. The existing EC2 doesn't have enough memory to run this new task, which implies that more number of EC2 instances are required to run the required tasks. This sets the value of CapacityProviderReservation more than 100. This will trigger the EC2 AutoScaling alarm which will increase the value of Desired EC2 Instances by 1 and then starts a new instance. Then that new provisioned task will run on this new EC2 Instance.

Later we will remove the traffic on our nginx server using that tool which will trigger other 2 CloudWatch Alarms and kill the tasks and EC2 instances to bring their number to minimum defined value. Thats how whole cluster automatically scales.

Lets start with downloading the tool Apache JMeter which will create traffic. You can follow this article to know how to use this tool. My configuration of this tool looks like following:

Thread Group

HTTP Request Defaults

HTTP Requests

View Results in Table

Now click on Run->Start in the Menu and generate traffic and you will see the Results in Results table:

Traffic Generated

Now open CloudWatch and click on Container Insights and Select Performance Monitoring->ECS Services. When the traffic reaches your container, value of ECSServiceAverageCPUUtilization starts increasing and soon it crosses the threshold 70 percent which is visible in the screenshot below in CPU Utilization graph. Also please note that, Number of Desired Tasks is 1 right now:

When the ECSServiceAverageCPUUtilization stays above 70 percent for 3 minutes it triggers the Service Autoscaling alarm which will increase the value of Desired Tasks by 1.

You can notice the value of Desired Tasks is set to 2 now and CPU Utilization graph is still on 100.

Now the Service starts provisioned the new task but this new task cannot be placed on the existing EC2 Instance because initially total available memory on that EC2 instance was 982Mb which is enough only for 1 task .Because for each task 512Mb memory should be available on the EC2 Instance to run it.

This concludes that more number of EC2 instances are required to run required tasks by the Service which increases the value of CapacityProviderReservation more than 100 (200 to be precise). This will trigger the EC2 AutoScaling Alarm. This will increase the value of Desired EC2 Instances by 1 and then starts a new instance.

As soon as the EC2 Instance becomes Active and the task is placed on that EC2 Instance, CapacityProviderReservation comes back on 100.

CPU Utilization also goes down because now incoming traffic is distributed among 2 containers by the load balancer.

Same can be seen in Container Insights in CloudWatch. You can also note that Number of Running tasks is 2 now.

But this peaceful state was momentarily because the 2 tasks together is still not sufficient for the incoming traffic and we see CPU Utilization of the Service again going up.

When the ECSServiceAverageCPUUtilization stays above 70 percent for 3 minutes it again triggers the Service Autoscaling alarm which will again increase the value of Desired Tasks by 1.

You can notice the value of Desired Tasks is set to 3 now and CPU Utilization graph is still on 100.

After the cooldown period ends, the Service starts provisioned the new task but again this new task cannot be placed on the existing EC2 Instance because of insufficient memory.

This concludes that more number of EC2 instances are required to run required tasks by the Service which again increases the value of CapacityProviderReservation more than 100. This will trigger the EC2 AutoScaling Alarm which will increase the value of Desired EC2 Instances by 1 and then starts a new instance.

As soon as the EC2 Instance becomes Active and the task is placed on that EC2 Instance, CapacityProviderReservation comes back on 100.

CPU Utilization also goes down because now incoming traffic is distributed among 3 containers by the load balancer.

Same can be seen in Container Insights in CloudWatch. You can also note that Number of Running tasks is 3 now.

Same scaling-out will happen one more time and our Cluster will settle down with one 4 tasks and 4 EC2 Instances:

Overall Container Insights for Service looks like this:

If you select ECS Tasks under Performance Monitoring then you will see same CPU Utilization because our service contains only one task definition.

Same can be seen if you select ECS Clusters under Performance Monitoring because our Cluster contains only one Service.

Now we will stop the traffic from Apache JMeter by clicking on Run->Stop . This will cause the CPU Utilization going close to 0 which will trigger other Service AutoScaling alarm which is responsible for scaling-in. That alarm will cause tasks getting stopped.
Stopped tasks will cause CapacityProviderReservation going less than 100 which will trigger other EC2 AutoScaling alarm which is responsible for scaling-in. That alarm will cause EC2 instances shutting down.

So that’s all about AWS ECS. But one more thing is remaining and that is automatic deployment of ECS. In the next and final article of the series , I am going to explain an efficient pipeline to deploy on ECS.