Monitoring Memory Leaks with AWS CloudWatch

Subscribe to my newsletter and never miss my upcoming articles

I recently found a memory leak in one of my applications. This motivated me to write a post about how you can profile a Spring Boot web app using Visual VM. Today, I will be discussing how I initially found this leak using Amazon CloudWatch and what steps I took to prevent unexpected mayhem from occurring!

AWS CloudWatch

First off — what is Amazon CloudWatch? CloudWatch is the control center for monitoring all your AWS resources in one place. You should regularly monitor your applications to ensure there are no surprise outages.

After I wake up I prepare breakfast and a coffee while I prepare for my daily standup meeting. In doing so, I normally log into our AWS account and take a look at CloudWatch to see traffic and look for anything out of the ordinary. CloudWatch is an Amazon service you can use to monitor AWS resources.

We had recently deployed a new containerized application to AWS Elastic Container Service (ECS). I noticed a few unusual metrics with the new application. The CPU seemed to be very active despite not be used and the memory utilization was increasing near 90% then diving back down.

0.-Found-Leak-in-Cloud-Watch-1.png These are not good metrics!

Immediately, I knew that something was not right. The app was frequently crashing which was causing the memory and CPU dips. Therefore, I set up a CloudWatch alarm with an SNS alert to email my team when the CPU spiked higher than 60%. In this case, someone could manually restart the application to reset the memory to prevent the application from crashing unexpectedly. This would be a bandaid until we found a proper solution. I documented the steps needed to create such an alarm.

Setup a CloudWatch Alarm

  1. Create an alarm from AWS CloudWatch 1.-Create-Button.png

  2. Select a metric for the alarm 2.-Select-Metric.png

  3. Select a metric category. Here I am going to use an ECS serviceName metric. 3.-Select-Memory.png

  4. Select “MemoryUtilization” for your category 4.-Select-Cluster.png

  5. I set the alarm to go off if the average Memory Utilization was > 60% Screen-Shot-2020-08-20-at-2.31.11-AM.png

  6. Use an SNS topic to send you or your team an email 6.-Config-2.png

  7. Make sure to add a description to keep track of different alarms 7.-Config-3.png

  8. Finally hit create! 8.-Create.png

Setup a CloudWatch Dashboard

Next, I will show you how to create a dashboard so you can take a look for yourself without setting up an alarm! CloudWatch Dashboards are very useful to get metrics for a variety of metrics across different services all in one place. I personally create a dashboard for each environment for a client: develop, QA, UAT, staging, production, etc. You first need to navigate to CloudWatch in your AWS account.

  1. Navigate to the dashboard link in CloudWatch and create a description 1.-Create-Dashboard.png

  2. Select a type of graph 2.-Select-type-of-graph.png

  3. Select a metric. Here I am using MemoryUtilization, but CloudWatch has a huge set of metrics 3.-Select-Metric.png

  4. Finally save the Dashboard! 4.-Save-Dashboard.png

Conclusion

I hope you enjoyed this post! Next, I will be writing a post describing exactly how I fixed this memory leak. Tune in and make sure to read my post about how to profile a Spring Boot application using Visual VM.

No Comments Yet