Make sure you have the following installed on your machine:

• make
• gnuplot
• Erlang/OTP

$ make all
or just
$ make
to omit the web interface. If you want to add that later, see the next section.

The web interface requires OTP R16B03-1 or later.

$ make ui

You have to build the web interface again if you move your bencherl folder:

$ make clean-ui
$ make ui

Specify what you want to run and how in conf/run.conf, and then use bencherl to run the benchmark suite.

$ ./bencherl

Use the -m option of the bencherl script.

$ ./bencherl -m everything-but-big

Set the INCLUDE_BENCH variable in conf/run.conf, if you want to specify which benchmarks to run.

INCLUDE_BENCH=bang,big

Set the EXCLUDE_BENCH variable in conf/run.conf, if you want to specify which benchmarks not to run.

EXCLUDE_BENCH=dialyzer_bench

The values of both variables are one or more benchmark names separated with commas.

By default, all benchmarks are run.

Use the -l option of the bencherl script.

$ ./bencherl -l

Set the NUMBER_OF_SCHEDULERS variable in conf/run.conf.

The value of this variable can be either one or more integers separated with commas:

NUMBER_OF_SCHEDULERS=1,2,4,8,16,32,64

or a range of integers:

NUMBER_OF_SCHEDULERS=1..16

By default, benchmarks are run with as many schedulers as the number of logical processors.

Set the OTPS variable in conf/run.conf.

The value of this variable is one or more alias-path pairs separated with commas.

OTPS="R14B04=/usr/local/otp_src_R14B04,R15B01=/usr/local/otp_src_R15B01"

By default, benchmarks are compiled and run with the erlc and erl programs found in the OS path.

Set the ERL_ARGS variable in conf/run.conf.

The value of this variable is one or more alias-arguments pairs separated with commas.

ERL_ARGS="SOME_ARGS=+sbt db +swt low,SOME_OTHER_ARGS=+sbt u"

Set the NUMBER_OF_SLAVE_NODES variable in conf/run.conf.

The value of this variable can be either one or more integers separated with commas:

NUMBER_OF_SLAVE_NODES=1,2,4,6,8

or a range of integers:

NUMBER_OF_SLAVE_NODES=2..4

Benchmarks are executed with at most as many slave nodes as specified in the SLAVE_NODES variable.

By default, benchmarks are run with one master node and no slave nodes.

Set the SLAVE_NODES variable in conf/run.conf.

The value of this variable is zero or more long or short node names separated with commas.

SLAVE_NODES=somenode@somehost,someothernode@someotherhost

The USE_LONG_NAMES variable determines whether long or short names are expected.

By default, benchmarks are run with no slave nodes.

Set the MASTER_NODE variable in conf/run.conf.

The value of this variabe is the short or the long name of the master node.

MASTER_NODE=somenode@somehost

The USE_LONG_NAMES variable determines whether long or short names are expected.

The default long name of the master node is:

master@`hostname -f`

and its default short name:

master@`hostname`

Set the COOKIE variable in conf/run.conf.

COOKIE="some_cookie"

The default cookie is cookie.

Set the VERSION variable in conf/run.conf.

The value of this variable can be short, intermediate or long.

VERSION=short

The default version is short.

Set the PLOT variable in conf/run.conf.

The value of this variable can be either 0 (do not produce any scalability graphs) or 1 (produce scalability graphs).

PLOT=1

The default value is 1.

Set the CHECK_SANITY variable in conf/run.conf.

The value of this variable can be either 0 (do not perform sanity check) or 1 (perform sanity check).

SANITY_CHECK=1

By default, the sanity of the benchmark execution results is not checked.

Set the ITERATIONS variable in conf/run.conf.

The value of this variable is an integer that is greater than or equal to 1.

ITERATIONS=5

The default number of iterations is 1.

By default, only the minimum runtime of any of the runs is reported. Set the OUTPUT_FORMAT variable in conf/run.conf to change this behaviour.

The value of this variable can be any of the following five options: min reports only the minimum runtime max reports only the maximum runtime avg reports the arithmetic mean of all iterations' runtimes avg_min_max reports all three of the above, in the order avg, min, max plain reports the runtime of each iteration individually

Note that the web interface only shows the first reported value and ignores any further numbers.

A new directory is created under the results directory. The name of this directory is the mnemonic name or, if no mnemonic name has been specified, a string that contains the date and time when the run started.

In the result directory, there is one subdirectory for each one of the benchmarks that was run, with the same name as the benchmark. Each such directory has three sub-directories:

• graphs, which contains the scalability graphs;
• output, which contains the output that the benchmark produced during its execution;
• measurements, which contains the scalability measurements collected during the execution of the benchmark .

The web interface can be used to view graphs for benchmark results stored in the results folder.

Use bencherlui with the -u (up) option to start the web server.

$ ./bencherlui -u

By default the web interface can be found at "http://localhost:8001".

You can change the port of the web server in ui/bencherlui/boss.config.

Use bencherlui with the -d (down) option to stop the web server.

$ ./bencherlui -d