Includes benchmarks for SHA256, AVL Tree, Base64 encoding, matrix mulitplication and finding the max element in an array.

For the newer benchmarks (not working at the moment due to limitations of Keystone) - Clone into the SDK directory. Run make and copy binaries into the bin folder.

In addition to benchmarks, we have analysis (see the Jupyter Notebook) on multiple runs of Keystone with an error bound on each benchmark. Using the plotly API, profiling the charts is possible.

Set up your firesim manager for running Keystone with the instructions here if you haven't yet.

Replace the tracerv_cc.c file with the one in this repo, located in firesim/sim/src/main/cc/endpoints.

Currently, our trace parsing scripts lie in here, so you'll want to checkout your firesim repo by doing:

git remote add upstream [email protected]:keystone-enclave/keystone-firesim.git

Or for HTTPs users:

git remote add upstream https://github.com/keystone-enclave/keystone-firesim.git

git fetch
git checkout cs294-keystone

For now, you must run your programs in Firesim one-by-one. After running one program, you'll need to transfer the TRACEFILE and extract the symbols from it for usage in the Jupyter Notebook.

scp <firesim-f1-instance>:~/sim_slot_0/TRACEFILE ~/firesim/trace
cd ~/firesim/trace
./list_symbols.sh
./parse_symbols.sh
python filterSymbols.py symbols.txt symbol.list symbols.filtered.txt
python ./analyze.py TRACEFILE symbols.filtered.txt output.txt

Note that there might be an IndexError after running the analyze.py script, which should be fine.

Now you can run through the Jupyter Notebook and get fancy charts.

pip install pandas matplotlib jupyter
pip install plotly cufflinks --user

jupyter notebook --no-browser --port=8080

ssh -i firesim.pem -N -L 8080:localhost:8080 <manager host>@<manager ip>

make

make clean