Understanding CPU Utilization and Credit Usage in AWS: A DevOps Perspective
In cloud computing and DevOps, optimizing resource use is vital for performance and cost control. AWS provides scalable infrastructure, but understanding metrics like CPU utilization, credit usage, and credit balance is key, particularly with burstable performance instances. This article highlights the importance of monitoring these metrics during high-concurrency load testing.
I’ll demonstrate this by using Apache Benchmark to test a T2 Micro AML2 instance with 10,000 concurrent requests on an HTTP server.
CPU Utilization
CPU utilization is a key metric that tracks the percentage of compute units in use on an instance. In AWS, monitoring this helps with right-sizing instances for optimal performance. High CPU utilization can indicate an overloaded instance, while low utilization might mean you’re over-provisioned.
Here is my Grafana dashboard showing the CPU utilization chart during two requests to my HTTP server. The spike in CPU utilization was due to the HTTP server software installation. We’ll revisit this chart after creating an AMI from this server.
When running load tests with tools like Apache Bench (ab) using 10,000 concurrent connections, monitoring CPU utilization is crucial. It helps identify performance bottlenecks and assesses whether your instance can effectively handle the simulated load.
Credit Usage and Credit Balance
AWS offers burstable performance instances (T2, T3, and T4g) that utilize a credit system, allowing instances to exceed their baseline CPU performance when needed. This extra performance is fueled by CPU credits, which are consumed when the instance operates above its baseline. The credit balance reflects the number of earned CPU credits, accumulated when CPU usage is below the baseline, based on the instance size.
In scenarios like load testing with 10,000 concurrent connections using Apache Bench, these instances can rapidly deplete their credit balance. It’s crucial for DevOps engineers to monitor credit usage closely to avoid unexpected performance drops once credits are exhausted.
Reproduce with this command
ab -n 10000 -c 100 http://instance-ip/ 
The command uses the Apache Bench utility in Unix to send 10,000 requests to the server with a concurrency of 100. During this process, you can observe the server’s CPU utilization and track how many credits are consumed. Metrics are collected at 5-minute intervals for accurate monitoring.
The chart clearly shows that CPU credits were consumed during the spike in CPU utilization.
Here’s a breakdown:
- During the Apache HTTP server installation, the average CPU utilization peaked at 9.91%, and five minutes later, the average credit usage was recorded at 0.498.
- When handling the spike in requests, CPU utilization reached 4.47%, while the average credit usage was 0.221.
Setting the records
Each burstable performance instance continuously earns credits when it stays below the CPU baseline, and continuously spends credits when it bursts above the baseline. The amount of credits earned or spent depends on the CPU utilization of the instance:
If the CPU utilization is below baseline, then credits earned are greater than credits spent.
If the CPU utilization is equal to baseline, then credits earned are equal to credits spent.
If the CPU utilization is higher than baseline, then credits spent are higher than credits earned.
Understanding the metric
1 CPU Credit equals 1 vCPU running at 100% utilisation for one minute.
The t2.micro is a burstable performance instance type in AWS, designed for workloads that don’t need consistently high CPU performance but may require occasional bursts. Let’s analyze its characteristics:
CPU Credits and Baseline Performance:
- Earns 6 CPU credits per hour
- Can accrue up to 144 credits
- Has 1 vCPU with a 10% baseline utilization
What this means:
- The instance can use 10% of its CPU continuously without consuming credits.
- Any CPU usage above 10% consumes credits from the balance.
- At full utilization (100% CPU), it would consume 5.4 credits per hour (90% above baseline * 6 credits).
Now, let’s stop the instance and create an AMI from it so that our new instance will include the HTTP server package.
We’ll then run the Apache Bench (ab) command on the new instance to monitor the metrics.
From the metrics dashboard, we can observe that the only spike in CPU utilization was due to the load test. There are no data points related to any installations, as the instance was created from the AMI with the HTTP server already installed.
From the graph, we see that peak CPU utilization occurred precisely at 19:10, reaching 3.36%.
Note that CloudWatch metrics are collected at 5-minute intervals, so the data points for 19:10 correspond to the interval ending at 19:00, and the 19:15 data point is essentially a continuation of 19:10.
At 19:15, CPU credit utilization peaked, averaging 0.168.
Through our analysis, we demonstrated how monitoring CPU utilization and credit usage is crucial during high-concurrency load testing. The use of Apache Bench (ab) to generate 10,000 concurrent requests highlighted the importance of these metrics. We observed that CPU credits were consumed during spikes in CPU utilization, and monitoring these metrics helps in understanding instance performance and preventing unexpected degradation.
By creating an AMI with the HTTP server pre-installed, we eliminated installation-related spikes and focused solely on the load test results. The data revealed that peak CPU utilization and credit usage aligned with the load test, providing clear insights into how burstable instances handle high demand.
Effective monitoring and understanding of CPU utilization and credit consumption are key to optimizing AWS instance performance and ensuring efficient resource management.
Thanks for Reading.
