EC2 Performance Benchmarking Guide¶

Proper benchmarking is essential for selecting the right EC2 instance types and optimizing application performance. This guide covers comprehensive benchmarking strategies.

Benchmarking Methodology¶

1. Define Your Workload¶

Before benchmarking, clearly define: - Application type: Web server, database, batch processing - Performance requirements: Latency, throughput, concurrent users - Resource usage patterns: CPU, memory, storage, network

2. Choose Representative Tests¶

Select benchmarks that mirror your actual workload characteristics.

CPU Benchmarks¶

Sysbench CPU Test¶

# Install sysbench
sudo yum install sysbench -y  # Amazon Linux
sudo apt install sysbench -y  # Ubuntu

# Single-threaded test
sysbench cpu --cpu-max-prime=20000 --threads=1 run

# Multi-threaded test  
sysbench cpu --cpu-max-prime=20000 --threads=$(nproc) run

UnixBench¶

# Download and run UnixBench
wget https://github.com/kdlucas/byte-unixbench/archive/v5.1.3.tar.gz
tar -xzf v5.1.3.tar.gz
cd byte-unixbench-5.1.3/UnixBench
make
./Run

Memory Benchmarks¶

Memory Bandwidth (Stream)¶

# Compile and run STREAM benchmark
gcc -O3 -mcmodel=medium -fopenmp stream.c -o stream
export OMP_NUM_THREADS=$(nproc)
./stream

Memory Latency (LMbench)¶

# Install and run lat_mem_rd
sudo yum install lmbench -y
lat_mem_rd 1024 128

Storage Benchmarks¶

Sequential I/O with fio¶

# Install fio
sudo yum install fio -y

# Sequential read test
fio --name=seqread --rw=read --bs=1M --size=4G --numjobs=1 --runtime=60

# Sequential write test  
fio --name=seqwrite --rw=write --bs=1M --size=4G --numjobs=1 --runtime=60

# Random I/O test
fio --name=randio --rw=randrw --bs=4k --size=1G --numjobs=4 --runtime=60

Database-like Workload¶

# Simulate database workload
fio --name=dbsim --rw=randrw --rwmixread=75 --bs=8k --size=2G --numjobs=8 --runtime=300

Network Benchmarks¶

iperf3 Network Throughput¶

# Server side
iperf3 -s

# Client side  
iperf3 -c <server-ip> -t 60 -P 4

Latency Testing¶

# Install hping3
sudo yum install hping3 -y

# Test latency
hping3 -c 100 -i 1 <target-ip>

Application-Specific Benchmarks¶

Web Server Performance¶

# Apache Bench
ab -n 10000 -c 100 http://your-server/

# wrk - modern HTTP benchmarking tool  
wrk -t12 -c400 -d30s http://your-server/

Database Performance¶

# sysbench MySQL test
sysbench oltp_read_write --mysql-host=localhost --mysql-user=test --mysql-password=test --mysql-db=testdb --tables=10 --table-size=100000 prepare

sysbench oltp_read_write --mysql-host=localhost --mysql-user=test --mysql-password=test --mysql-db=testdb --tables=10 --table-size=100000 --threads=16 --time=300 run

Benchmark Results Interpretation¶

CPU Performance Metrics¶

• Single-thread performance: Critical for legacy applications
• Multi-thread scalability: Important for parallel workloads
• Instructions per cycle (IPC): Measure of CPU efficiency

Memory Performance Metrics¶

• Bandwidth: GB/s throughput for streaming workloads
• Latency: ns access time for random access patterns
• Cache efficiency: Hit rates at different cache levels

Storage Performance Metrics¶

• IOPS: Input/output operations per second
• Throughput: MB/s bandwidth
• Latency: Response time (p95, p99 percentiles)

Network Performance Metrics¶

• Throughput: Gbps maximum bandwidth
• Latency: Round-trip time
• Packet loss: Reliability under load

Comparing Instance Types¶

Create a standardized test suite and run across different instance types:

#!/bin/bash
# benchmark-suite.sh

echo "=== CPU Benchmark ==="
sysbench cpu --cpu-max-prime=20000 --threads=$(nproc) run | grep "total time"

echo "=== Memory Benchmark ==="
sysbench memory --memory-total-size=10G --threads=$(nproc) run | grep "total time"

echo "=== Disk Benchmark ==="  
fio --name=test --rw=randrw --bs=4k --size=1G --numjobs=4 --runtime=60 --output-format=json | jq '.jobs[0].read.iops, .jobs[0].write.iops'

echo "=== Network Benchmark ==="
# Requires separate server instance
# iperf3 -c <server-ip> -t 30 | grep receiver

Performance Monitoring During Production¶

CloudWatch Metrics¶

Monitor key metrics: - CPU Utilization - Memory Usage (with CloudWatch Agent) - Disk I/O (IOPS, throughput) - Network I/O (packets, bytes)

Application Performance Monitoring¶

Use tools like: - New Relic: Application performance insights - DataDog: Infrastructure and application monitoring
- Prometheus: Open-source monitoring with Grafana

Best Practices¶

1. Benchmark Consistently¶

• Use same AMI across tests
• Run tests multiple times
• Control for external factors (time of day, network conditions)

2. Test Realistic Workloads¶

• Mirror production data patterns
• Include peak load scenarios
• Test failure recovery performance

3. Consider Total Cost of Ownership¶

Balance performance gains against cost increases:

Cost per unit of performance = (Instance cost per hour) / (Benchmark score)

Conclusion¶

Effective benchmarking requires a systematic approach tailored to your specific workload. Use multiple benchmarks, test consistently, and always validate results with production workloads.