This repository contains Ansible and R scripts to run benchmarks on web servers and plot the data.
There are two Ansible playbooks, one to install the servers and the other one to run the benchmarks, collect and process the data.
The hosts.ini file contains the configurations of the hosts. There is a dev environment by using vagrant and a prod environment by using hetzner.
The dev environment uses Vagrant to test the playbooks, a Vagrantfile is available in the repository to create the VMs.
The prod environment needs additional settings like specifying the IP addresses and the SSH key file.
The environment can be changed by adding --extra-vars "env=prod".
By default, the environment is set to dev.
This playbook is used to install the web server, copy their configurations and the static files.
The command in below will install the server Apache on the prod environment.
ansible-playbook -i hosts.ini playbooks/install_server/main.yml --extra-vars "server={{apache}} env=prod" --forks 15The available servers can be viewed, added and deleted in servers-default.yml.
This playbook installs Apache Benchmark, runs the benchmark, process the data to a CSV file and copies the files to the local machine.
The command in below runs in the dev environment, stops all the web servers, starts Nginx and runs the benchmark against it with a 10000 number of requests, 200 concurrency to the static file of 20KB size. It also skips the cloning of the repository containing the Python script to process the ab data to a CSV file.
ansible-playbook -i hosts.ini playbooks/ab/main.yml --extra-vars "server={{nginx}} n=10000 c=200 path=/static-20k.html skip_clone=True" --forks 15At the end of the run, in playbooks/ab/out/{ip}/out.csv there will be the csv file containing
the collected data. Additional hardware data is added to the CSV file,
in extra_data can be found which data is added.
| server | host | port | path | doc_len | conc | time | r_comp | r_fail | rps | tpr | tpr_all | 50 | 66 | 75 | 80 | 90 | 95 | 98 | 99 | 100 | iso8601 | os | arc | memtotal_mb | disk | pcores | vcpus |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Apache/2.4.52 | 127.0.0.1 | 80 | /static-20k.html | 20480 | 200 | 3.077 | 50000 | 0 | 16248.86 | 12.309 | 0.062 | 12 | 12 | 13 | 13 | 15 | 16 | 19 | 21 | 48 | 2023-12-08T11:00:38Z | Ubuntu_22.04 | x86_64 | 31334 | 362633863168 | 16 | 16 |
| nginx/1.18.0 | 127.0.0.1 | 80 | /static-20k.html | 20480 | 200 | 2.876 | 50000 | 0 | 17387.81 | 11.502 | 0.058 | 11 | 12 | 12 | 13 | 14 | 16 | 18 | 19 | 24 | 2023-12-08T11:07:50Z | Ubuntu_22.04 | x86_64 | 31334 | 362633863168 | 16 | 16 |
| lighttpd/1.4.63 | 127.0.0.1 | 80 | /static-20k.html | 20480 | 200 | 2.457 | 50000 | 0 | 20349.14 | 9.828 | 0.049 | 9 | 10 | 10 | 10 | 12 | 14 | 17 | 21 | 26 | 2023-12-08T11:13:44Z | Ubuntu_22.04 | x86_64 | 31334 | 362633863168 | 16 | 16 |
| Caddy | 127.0.0.1 | 80 | /static-20k.html | 20480 | 200 | 2.863 | 50000 | 0 | 17463.82 | 11.452 | 0.057 | 11 | 11 | 12 | 13 | 14 | 16 | 17 | 18 | 72 | 2023-12-08T11:21:53Z | Ubuntu_22.04 | x86_64 | 31334 | 362633863168 | 16 | 16 |
| LiteSpeed | 127.0.0.1 | 80 | /static-20k.html | 20480 | 200 | 2.537 | 50000 | 0 | 19708.62 | 10.148 | 0.051 | 9 | 10 | 11 | 11 | 14 | 16 | 18 | 19 | 26 | 2023-12-08T11:28:53Z | Ubuntu_22.04 | x86_64 | 31334 | 362633863168 | 16 | 16 |
plots.R contains the script to generate the plots. The documentation for it is added as comments in the code.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent