These scripts can be used to automate measurements of different protocols on the OpenSAND satellite emulation platform. Each emulation (one execution of the opensand.sh script) consist of multiple scenarios that are tested, each of which has a different configuration (such as the orbit of the emulated satellite). Within a single scenario multiple measurements are executed to measure the performance of different protocols. Each measurement is executed multiple times with each execution being called a run. This will generate more statistically stable results.

This is an overview of the environment that is create by the scripts using network namespaces.

While the complete output of each component taking part in a measurement is captured, the runs aim to measure the following set of performance values:

• Goodput Evolution
• Congestion Window Evolution
• Connection Establishment Time
• Time to First Byte
• Web Performance Metrics (e.g., Response Start, First Contentful Paint, Page Load Time) for HTTP/1.1 and HTTP/3

For evaluation of the result you can use these scripts: quic-opensand-evaluation

The main executable script is opensand.sh it will source all other scripts before starting the measurements. Some scripts (such as setup.sh and teardown.sh) can also be executed individually for e.g. manual measurements.

• opensand.sh - Main executable
• env.sh - Environment configuration
• setup*.sh - Environment creation and setup
• teardown*.sh - Environment disassembly
• run*.sh - Execution of the individual measurement runs
• stats.sh - System statistics collection during the emulation

1. Ensure the requirements below are met
2. Copy all files (including subdirectories) to the machine that will run the emulation
3. Update configuration in env.sh, especially the file and directory paths

These programs need to be cloned and built

• qperf
• pepsal

The following utilities need to be installed on the system:

• opensand (which installs opensand-core, opensand-network and opensand-daemon)
  Not required are opensand-collector and opensand-manager
• iperf3
• tmux
• curl
• nginx (deamon can be disabled, it is used standalone)
• iproute2
• xmlstarlet

In addition, the following utilities are required for the Web Performance Measurements:

• chromium
• selenium
• h2o Web Server

Executing the main script opensand.sh will start the automated emulation. As this will take some time, it is recommended to start the script in a tmux session.

tmux new-session
./opensand.sh

This allows to detach from the process and re-attach at any time later.

The results of an emulation can be found in a subdirectory of the configured RESULTS_DIR (set in env.sh), along with the emulation log file. To simplify downloading the results, the symlink latest in RESULTS_DIR is updated to the latest emulation output directory. When downloading the results, it is recommended to use rsync over scp since the output consists of many small files.

The script can be interrupted at any point, which will stop the current emulation and cleanup the environment.

These parameters configure the scenarios that are executed. All combinations of all configured values are executed. Please see Scenario file format and Example file for a more fine-grained control over the individual scenarios and example configurations.

The command line arguments are used to generate a temporary scenario configuration file in the emulations temporary directory (/tmp/opensand.*/).

The scenario file allows a much more fine-grained control over the individual scenarios that are executed. Each line in the file describes a single scenario. Blank lines and lines starting with # are ignored. For each scenario the exact same arguments and syntax are used as for the scenario configuration command line arguments with the exception, that only a single scenario must be described. Repeatable arguments must only be given once. Arguments that define different configuration values via comma separated lists must only have a single value.

# Example scenario configuration

-N 100 -T 10 -P 5 -A 0 -B 4M,4M -Q 4M,4M,4M,4M -U 4M,4M,4M,4M -l client.qlog,server.qlog -O GEO -E 125,125 -W -C cccc -L 0 -I 10,10,10,10

-N 100 -T 10 -P 5 -A 0 -B 4M,4M -Q 4M,4M,4M,4M -U 4M,4M,4M,4M -l client.qlog,server.qlog -O LEO -E 8,8 -C -V crrc -L 0.1 -I 10,100,100,10

This file describes 2 different scenarios. Both scenarios perform 100 iterations of goodput and http measurements (-N 100), and 10 iterations of timing measurements (-T 10). The environments are primed for 5 seconds (-P 5), and the attenuation is configured with 0db (-A 0). All Buffers are identically sized in both scenarios with 4MB, and are also identical for the different buffer types (-B 4M,4M -Q 4M,4M,4M,4M -U 4M,4M,4M,4M). The client and server qlogs are output to the files client.qlog and server.qlog (-l client.qlog,server.qlog()

The first scenario uses a GEO satellite orbit (-O GEO) with a one-way delay of 250ms (-E 125,125), disables PEP measurements (-W), and uses cubic as congestion control on server, gateway, terminal, and client (-C cccc). Moreover, the loss is configured to 0% (-L 0), and the initial window set to 10 on server, gateway, terminal, and client (-I 10,10,10,10)

The second scenario uses a LEO satellite orbit (-O LEO) with a one-way delay of 16ms (-E 8,8), disables Non-PEP measurements (-V), and uses cubic as congestion control on server and client, as well as reno on gateway and terminal (-C crrc). Moreover, the loss is configured to 0.1% (-L 0.1), and the initial window set to 10 on server and client, as well as 100 on gateway and terminal (-I 10,100,100,10).