On the client node the fastswap kernel and driver must be loaded. On the far memory node the server binary rmserver must be running. Please see https://github.com/clusterfarmem/fastswap for more details.

You'll need python3, grpcio, grpcio-tools, numpy and scipy to execute various parts of our framework. Please make sure your python environment can see these modules.

• quicksort
  • Change directory to quicksort and type make
• linpack
  • No setup required, but most likely you'll need an Intel CPU
• tf-inception
  • tensorflow 1.14 is required
  • Init submodules git submodule update --init
• spark
  • We assume the user has installed spark 2.4 at ~/spark-2.4.0-bin-hadoop2.7
• kmeans
  • Requires sklearn available in python3
• memcached
  • Requires memcached and memaslap to be installed and available in your $PATH environment.
• stream
  • Change directory to stream and type make

• Open /boot/grub/grub.cfg in your editor of choice
• Find the menuentry for the fastswap kernel
• Add cgroup_no_v1=memory to the end of the line beginning in linux /boot/vmlinuz-4.11.0-sswap
• Save and exit the file
• Run: sudo update-grub
• Reboot

The framework and scripts rely on the cgroup system to be mounted at /cgroup2. Perform the following actions:

• Run sudo mkdir /cgroup2 to create root mount point
• Execute setup/init_bench_cgroups.sh
  • Mounts cgroup system
  • Changes ownership of the mount point (and all nested files) to the current user
  • Enables prefetching

We use the grpc framework and protocol buffers to communicate between the scheduler and servers. The messages that we've defined are in protocol/protocol.proto. To generate them the corresponding .py files, execute the following command in the protocol directory:

source gen_protocol.sh

Benchmark.py is the command center from which you can run local, single benchmarks. It accepts numerous arguments but only two, workload and ratio, are required. Its minimum invocation is the following:

./benchmark.py <workload> <ratio>

Where workload is an application that the toolset has been configured to benchmark (Ex: linpack) and ratio is the portion of its resident set size that you want to keep in local memory, expressed as a decimal.

Running the tool in this way will set the appropriate limits in the applications cgroup, run it to completion, then print statistics to stdout.

./benchmark.py linpack 0.5 --cpus 4,5,6,7

./benchmark.py quicksort 0.3 --id 5

New workloads can be added by modifying the workload_choices variable in benchmark.py and creating a new class for it in lib/workloads.py.

server.py runs on a separate (or even the same) machine from scheduler.py. Multiple server.py instances send execution-related data to a single scheduler.py instance, receiving workload execution directions in turn. server.py takes a single, optional flag, --log, that directs it to save a timestamped account of events to a file named log.txt in the same directory.

We made a lot of assumptions about system configuration. server.py expects several files to exist on your system, mostly for sampling purposes. If they don't exist, we insert zeroes instead of reading their values.

This is the brains of the server-scheduler system. The scheduler is responsible for determining the arrival order of workloads, setting the shrinking policy, and aggregating all of the data from the server(s).

./scheduler.py 123 192.168.0.1:50051 8 8192 -r --max_far 4096 --size 100 \
--workload quicksort,kmeans,linpack --ratios 3:1:1 --until 30 --optimal

./scheduler.py 123 192.168.0.1:50051 8 8192 -r --size 100 --workload quicksort,kmeans,linpack \
--ratios 3:1:1 --until 30 --variable_ratios 0.5,0.6,0.7

./scheduler.py 123 192.168.0.1:50051,192.168.0.2:50051 8 8192 -r --size 250 \
--workload quicksort,kmeans,linpack --ratios 3:1:1 --uniform_ratio 0.5 \
--until 30 --start_burst 2

For more information, please refer to our paper accepted at EUROSYS 2020

For additional questions please contact us at [email protected]