VLab-Bench is a suite that offers benchmarks for real-world scientific design tasks and optimisation algorithms for materials science and biology.

The reconstructed phases (of the object transmission functions) with parameters obtained from the corresponding DFO methods on a MoS2 dataset.

Results are averaged over 5 trials, and ± denotes the standard deviation.

The code requires python>=3.9. Installation Tensorflow and Keras with CUDA support is strongly recommended.

Install vlab_bench:

pip install -e "git+https://github.com/poyentung/VLab-Bench.git"

or clone the repository to local devices:

git clone https://github.com/poyentung/VLab-Bench.git
cd vlab_bench; pip install -e ./

[Optional] Install TurBO and/or LaMCTS

git clone https://github.com/uber-research/TuRBO.git
pip install -e TuRBO/./

git clone https://github.com/facebookresearch/LaMCTS.git
pip install -e LaMCTS/LaMCTS/LA-MCTS/./

[Optional] Install py4DSTEM

pip install py4dstem

Or check installation for GPU acceleration.

We run parameter optimization for electron ptychography using TuRBO on a MoS2 dataset in 14 dimensions for 20 samples with 30 initial data points. Note that num_samples should include the init_samples for TuRBO and LaMCTS, i.e., num_samples=50 and init_samples=30 represent 20 aquisition of samples (50 - 30 = 20). More detailed hyper-parameters can be adjusted in the run_pytho.yaml.

python scripts/run_ptycho.py method=turbo \
                             func=ptycho \
                             dims=14 \
                             num_samples=50 \
                             init_samples=30

We evaluate TuRBO on Ackley in 10 dimensions for 1,000 samples with 200 initial data points. Note that num_samples should include the init_samples for TuRBO and LaMCTS, i.e., num_samples=1000 and init_samples=200 represent 800 aquisition of samples (1000 - 200 = 800). More detailed hyper-parameters can be adjusted in the run.yaml.

python scripts/run.py method=turbo \
                      func=ackley \
                      dims=10 \
                      num_samples=1000 \
                      init_samples=200

We can also run multiple conditions in a run. For example, we want to evaluate MCMC, CMA-ES and Dual Annealing on Ackley in 10 dimensions for 1,000 samples with 200 initial data points .

python scripts/run.py -m method=mcmc,cmaes,da \
                         func=ackley \
                         dims=10 \
                         num_samples=20 \
                         init_samples=200

• Cyclic peptite binder design
• Electron ptychography

Please send us a PR to add your real-world task!

• Ackley
• Rastrigin
• Rosenbrock
• Schwefel
• Michalewicz
• Griewank

• TurBO
• LaMCTS
• Dual Annealing
• Differential Evolution
• CMA-ES
• MCMC
• Shiwa
• DOO
• SOO
• VOO

Please send us a PR to add your algorithm!

The source code is released under the MIT license, as presented in here.