victorcq / aws-network-benchmark Goto Github PK

Network performance benchmarks for AWS Fargate and AWS Lambda.

Here are some results from benchmarks performed on eu-west-1 and eu-north-1 regions on 2020-05-17. See analysis/results.ipynb for more detailed graphs.

Observations:

• All tasks have very stable baseline performance.
  • Bigger tasks seem to have slightly faster network.
• Some tasks are able to burst up-to 10 Gbps.
  • Most tasks are not able to burst beyond baseline capacity.

Observations:

• Tasks with < 1 vCPU are able to burst to 5 Gbps.
  • Length of burst depends primarily on CPU allocation.
  • Length is a bit longer for containers with largest possible memory allocation for a vCPU configuration.
• Tasks with >= 1 vCPUs are able to burst to 10 Gbps
  • Length of burst depends on CPU & Memory allocation
  • Tasks with largest memory allocation get slightly longer burst.
  • Length of burst is shorter than that of smaller containers (but has 2x more bandwidth)
• Network performance is much better than in eu-west-1.
  • Burst capacity is available for all containers sizes.
  • Baseline capacity increases linearly as the container grows (except for the biggest containers)

Here are some results from benchmarks performed on eu-west-1 and eu-north-1 regions on 2020-06-16. See analysis/results-lambda.ipynb for more detailed graphs.

• Lambda functions have a stable 600 Mbps network throughput with a small burst to 1.7 Gbps in the beginning
• No differences between regions

You must have the following setup to be able to run these benchmarks:

• AWS credentials with admin level privileges
• AWS account with VPC and Subnets from sjakthol/aws-account-infra.
• Docker

Once setup, you can deploy the benchmark infra (ECR, ECS, Roles, Security Groups, Log Groups) by running

make deploy-infra

in this directory. Once the ECR registry has been created, you'll need to build the Docker image used for benchmarks:

make login build tag push

Use one of the benchmark targets from the Makefile to execute benchmarks:

make benchmark-fargate-512-1024 # benchmark of Fargate container with 0.5vCPU and 1GB memory

The targets will create a Fargate task of specified size, execute the benchmark and delete the resources used in the benchmark.

You must have the following setup to be able to run these benchmarks:

• AWS credentials with admin level privileges
• AWS account with VPC and Subnets from sjakthol/aws-account-infra.
• Docker

Once setup, you can deploy the benchmark infra (ECR, ECS, Roles, Security Groups, Log Groups) by running

make deploy-infra

in this directory. Once done, you'll need to prepare the Lambda function by running

(cd lambda && make lib)

Finally, you'll need to deploy the iperf3 server run running

make deploy-lambda-server

Use one of the benchmark targets from the Makefile to execute benchmarks:

make benchmark-lambda-512 # benchmark of Lambda function 512 MB of memory
make benchmark-lambda_all # benchmark different configs one by one

The targets will create a Lambda function of specified size, execute the benchmark and delete the resources used in the benchmark.

Delete the iperf3 server by running

make delete-lambda-server

This project uses iperf3 to measure network performance.

For Fargate, we package an iperf3 server into a Docker container, deploy the container as a Fargate task and create an EC2 instance to execute iperf3 client against the iperf3 server in Fargate. Benchmarks configure iperf3 client to use 10 connections to execute a 15 minute benchmark in reverse mode (server sends the data to the client, not the other way around).

For Lambda, we package an iperf3 client into a Lambda function, deploy an iperf3 server into EC2 and execute iperf3 client in Lambda against an iperf3 server in EC2. Benchmarks configure iperf3 client to use 1 connection to execute a 60 minute benchmark (client sends data to server).

Benchmark results are saved to Amazon S3 as JSON with the output from iperf3 as well as some custom metadata from the execution.

• https://github.com/widdix/ec2-network-benchmark

MIT