ENTMOOT
(ENsemble Tree MOdel Optimization Tool) is a framework to handle tree-based surrogate models in Bayesian optimization applications. Gradient-boosted
tree models from lightgbm
are combined with a distance-based uncertainty
measure in a deterministic global optimization framework to optimize black-box functions. More
details on the method here: https://arxiv.org/abs/2003.04774.
When using any ENTMOOT
for any publications please reference this software package as:
@article{thebelt2021entmoot,
title={ENTMOOT: A framework for optimization over ensemble tree models},
author={Thebelt, Alexander and Kronqvist, Jan and Mistry, Miten and Lee, Robert M and Sudermann-Merx, Nathan and Misener, Ruth},
journal={Computers \& Chemical Engineering},
volume={151},
pages={107343},
year={2021},
publisher={Elsevier}
}
The "ask-tell" optimizer interface for an intuitive incorporation of own black-box functions was included from scikit-optimize which is available under the BSD 3-Clause License.
- python >= 3.7
- numpy >= 1.18.4
- scikit-learn >= 0.21.3
- pyyaml >= 5.3.1
- lightgbm >= 2.3.1
On Mac, you will also need to install libomp: brew install libomp
.
- gurobi >= 9.0.1
Install all required packages by running the command
pip install -r requirements.txt
To install ENTMOOT
, run the following command
pip install git+https://github.com/cog-imperial/entmoot
The ENTMOOT package can be uninstalled by running
pip uninstall entmoot
The installation on Windows-based systems is similar with some exceptions for
individual commands. Make sure that python
and git
is installed and
accessible in the shell you are using to install ENTMOOT
. Follow the
instructions by executing all commands in cmd.exe
.
Install all required packages by running the command
pip install -r requirements.txt
To install ENTMOOT
, run the following command
pip install git+https://github.com/cog-imperial/entmoot
The ENTMOOT package can be uninstalled by running
pip uninstall entmoot
To use the acq_optimizer= 'global'
setting in ENTMOOT
, the solver
software Gurobi v9.0
or newer is required. Gurobi is a commercial mathematical optimization solver and
free of charge for academic research. It is available on Linux, Windows and
Mac OS.
Please follow the instructions to obtain a free academic license. Once Gurobi
is installed on your system, follow the steps to setup the Python interface gurobipy in your virtual environment created for ENTMOOT
.
For a simple example we use Rosenbrock as our black-box function. We import the
entmoot_minimize
function which automates the optimization procedure. The
black-box function is given as func
to the solver. For more details on
individual settings we refer to the source code which explains all parameters
in detail.
An example script that minimizes the Rosenbrock function using ENTMOOT
is
given in the following.
from entmoot.optimizer.optimizer import Optimizer
from entmoot.benchmarks import Rosenbrock
func = Rosenbrock()
opt = Optimizer(func.get_bounds(10),
base_estimator="ENTING",
n_initial_points=20,
initial_point_generator="random",
acq_func="LCB",
acq_optimizer="sampling",
random_state=100,
model_queue_size=None,
base_estimator_kwargs={
"lgbm_params": {"min_child_samples": 1}
},
verbose=True,
)
# run optimizer for 20 iterations
res = opt.run(func, n_iter=20)
print(f"-> best solution found {res.fun}")
The following example shows how own constraints can be enforced on the input
variables. This only works when Gurobi is selected as the solution strategy,
i. e. acq_optimizer="global"
. Constraints are formulated according to
documentation.
from entmoot.optimizer.optimizer import Optimizer
from entmoot.benchmarks import SimpleCat
func = SimpleCat()
# initialize the search space manually
from entmoot.space.space import Space
space = Space(func.get_bounds())
# get the core of the gurobi model from helper function 'get_core_gurobi_model'
from entmoot.optimizer.gurobi_utils import get_core_gurobi_model
core_model = get_core_gurobi_model(space)
# ordering of variable indices is dependent on space definition
# cont_var_dict contains all continuous variables
x0 = core_model._cont_var_dict[0]
x1 = core_model._cont_var_dict[1]
# cat_var_dict contains all categorical variables
x2 = core_model._cat_var_dict[2]
# define constraints accordingly
core_model.addConstr(x0 + x1 >= 2)
core_model.addConstr(x1 == 1)
core_model.update()
opt = Optimizer(func.get_bounds(),
base_estimator="ENTING",
n_initial_points=0,
initial_point_generator="random",
acq_func="LCB",
acq_optimizer="global",
random_state=100,
acq_func_kwargs=None,
acq_optimizer_kwargs={
"add_model_core": core_model
},
model_queue_size=None,
base_estimator_kwargs={
"lgbm_params": {"min_child_samples": 1}
},
verbose=True,
)
# add initial points that are feasible
for x in [(1.1, 1.0, 'mult6'), (1.2, 1.0, 'mult6'), (1.3, 1.0, 'pow2')]:
opt.tell(x, func(x))
# run optimizer for 20 iterations
res = opt.run(func, n_iter=20)
print(f"-> best solution found {res.fun}")
ENTMOOT
also supports multi-objective optimization according to:
@article{thebelt2022multi,
title={Multi-objective constrained optimization for energy applications via tree ensembles},
author={Thebelt, Alexander and Tsay, Calvin and Lee, Robert M and Sudermann-Merx, Nathan and Walz, David and Tranter, Tom and Misener, Ruth},
journal={Applied Energy},
volume={306},
pages={118061},
year={2022},
publisher={Elsevier}
}
An example that derives Pareto-optimal points of the Fonzeca Freming is given in the following:
from entmoot.benchmarks import FonzecaFleming
from entmoot.optimizer import Optimizer
# initialize multi-objective test function
funcMulti = FonzecaFleming()
# define optimizer object and specify num_obj=2
opt = Optimizer(funcMulti.get_bounds(),
num_obj=2,
n_initial_points=10,
random_state=100)
# main BO loop that derives pareto-optimal points
for _ in range(50):
next_x = opt.ask()
next_y = funcMulti(next_x)
opt.tell(next_x,next_y)
Using multi-objective functionality in ENTMOOT
requires the specification of num_obj
which
informs the solver about the number of objectives that we optimize for. ENTMOOT
minimizes
objectives which requires the modification of maximization problems, i.e. minimizing the
negative objective.
- Alexander Thebelt (ThebTron) - Imperial College London
- Nathan Sudermann-Merx (spiralulam) - Cooperative State University Mannheim
- David Walz (DavidWalz) - BASF SE
The ENTMOOT package is released under the BSD 3-Clause License. Please refer to the LICENSE file for details.
The support of BASF SE, Lugwigshafen am Rhein is gratefully acknowledged.