dme65 / pysot Goto Github PK
View Code? Open in Web Editor NEWSurrogate Optimization Toolbox for Python
License: Other
Surrogate Optimization Toolbox for Python
License: Other
Hi there,
to be sure, currently the only one stopping criteria supported in pySOT is maximum number of evaluations, am I right?
The second question is about failed evaluation. What should my own "objfunction" method return to consider it as failed evaluation by pySOT? Should "ValueError" exception arise? And what about optimization process behavior in this case, does it stop or continue? (I've read the issues #8 and #9)
Following the latest documentation, I try to install it like this:
virtualenv .venv
. .venv/bin/activate
git clone https://github.com/dme65/pySOT
cd pySOT
pip install inspyred
python setup.py install
I get this error:
ImportError: No module named numpy.distutils.core
Running python setup.py install
again, for the second time, will complete the installation.
PySOT doing very well here versus Hyperopt, Optuna etc...
https://www.microprediction.com/blog/optimize
In the documentation it says:
CandidateDYCORS Uses a DDS strategy...
however after reading the code and the DYCORS paper (Regis & Shoemaker, 2012), the search strategy it's actually DYCORS-LMSRBF, not DYCORS-DDSRBF.
Please confirm.
Thanks
I have the three parameters (node1,node2,node2) with each parameters I have 20 values.
nodes1 = np.array([0.05675, 0.05934, 0.05633, 0.0557 , 0.05702, 0.06401, 0.06322, 0.06571, 0.06099, 0.05832, 0.06196, 0.06463, 0.05507, 0.06351, 0.06287, 0.06122, 0.05407, 0.05985, 0.05774,0.06015])
nodes2 = np.array([0.9486, 0.9095, 0.9856, 0.9318, 1.0477, 1.0489,1.0663, 0.9184, 0.9646, 1.0345, 1.0168, 1.0565, 0.9727, 0.9907, 0.9277, 0.9548, 1.0933, 1.0751,1.0026, 1.0231])
nodes3 = np.array([51.813, 54.279, 52.659, 51.197 , 46.629, 49.791, 48.581, 54.799, 46.413, 47.078, 52.367, 48.204, 50.389, 45.402, 47.893, 50.796 , 49.332, 53.323, 53.713, 45.757])
with a group parameter of input I have a output values. So I have the 20 output values
example: (node1_1,node2_1,node3_1)=(0.05675,0.9486,51.813) I have a output values = 0.0204232
output=np.array([0.0204232,0.0205054,0.0204971,0.0204463,0.0206686,0.0206678,0.0206883,0.0204627,0.020426,0.0206532,0.0206322,0.020677,0.0204431,0.0205319,0.0204508,0.0204115,0.020721,0.0206988,0.0206179,0.0206418])
How to estimate the surrogate model between (node1,node2,node3) with output? and how to know the detail equation of the surrogate model?
please help me, thanks you
Hi,
I run a simple example with num_threads = 1, it is OK but with
multiple threads=4, I got the following error. I use the same example code (example_mars()).
However I replace the objective function as:
import pySOT.optimization_problems as sprob
PORT=8500
url="http://localhost:" + str(PORT) + "pysot_test"
class PySOTTestProblem(sprob.OptimizationProblem):
def init(self):
self.dim=1
self.info = "PySOT test Problem" #optional
self.lb = 0.9np.array([0.0])
self.ub = 1.1np.array([2*np.pi])
self.int_var = np.array([])
self.cont_var = np.arange(0, 1)
def eval(self, x):
"""Evaluate the objective function at x
:param x: Data point
:type x: numpy.array
:return: Value at x
:rtype: float
"""
self.__check_input__(x)
print("x=", x)
spec=dict(x=x[0])
results = requests.post(url,json=spec, timeout=6400).json()
objective = float(results['objective'])
print("x=", beta, "objective=", objective)
return objective
==============================================
on the server side
I have a web server (gunicorn and falcon) running and return the value of np.sin(x).
ValueError Traceback (most recent call last)
in
----> 1 equity_model_adaptation()
in equity_model_adaptation()
38
39 # Run the optimization strategy
---> 40 result = controller.run()
41
42 print('Best value found: {0}'.format(result.value))
~/miniconda3/envs/dp36/lib/python3.6/site-packages/poap/controller.py in run(self, merit, filter)
342 """
343 try:
--> 344 return self._run(merit=merit, filter=filter)
345 finally:
346 self.call_term_callbacks()
~/miniconda3/envs/dp36/lib/python3.6/site-packages/poap/controller.py in _run(self, merit, filter)
312 self._run_queued_messages()
313 time.sleep(0) # Yields to other threads
--> 314 proposal = self.strategy.propose_action()
315 if not proposal:
316 self._run_message()
~/miniconda3/envs/dp36/lib/python3.6/site-packages/pySOT/strategy.py in propose_action(self)
279 self.phase = 2
280 if self.asynchronous: # Always make proposal with asynchrony
--> 281 self.generate_evals(num_pts=1)
282 elif self.pending_evals == 0: # Make sure the entire batch is done
283 self.generate_evals(num_pts=self.batch_size)
~/miniconda3/envs/dp36/lib/python3.6/site-packages/pySOT/strategy.py in generate_evals(self, num_pts)
562 opt_prob=self.opt_prob, num_pts=num_pts, surrogate=self.surrogate,
563 X=self.X, fX=self.fX, Xpend=self.Xpend, weights=weights,
--> 564 sampling_radius=self.sampling_radius, num_cand=self.num_cand)
565
566 for i in range(num_pts):
~/miniconda3/envs/dp36/lib/python3.6/site-packages/pySOT/auxiliary_problems.py in candidate_srbf(num_pts, opt_prob, surrogate, X, fX, weights, Xpend, sampling_radius, subset, dtol, num_cand)
106 """
107 # Find best solution
--> 108 xbest = np.copy(X[np.argmin(fX), :]).ravel()
109
110 # Fix default values
~/miniconda3/envs/dp36/lib/python3.6/site-packages/numpy/core/fromnumeric.py in argmin(a, axis, out)
1099
1100 """
-> 1101 return _wrapfunc(a, 'argmin', axis=axis, out=out)
1102
1103
~/miniconda3/envs/dp36/lib/python3.6/site-packages/numpy/core/fromnumeric.py in _wrapfunc(obj, method, *args, **kwds)
49 def _wrapfunc(obj, method, *args, **kwds):
50 try:
---> 51 return getattr(obj, method)(*args, **kwds)
52
53 # An AttributeError occurs if the object does not have
ValueError: attempt to get argmin of an empty sequence
==========end of error========================================
We recently tracked down a PendingDeprecationWarning
from our application test suite to the uses of np.asmatrix
in this repository. I can't post the actual traceback here, but here's an example of the same effect (Python 3.7.2, NumPy 1.15.4):
mdickinson$ python -Wd
Python 3.7.2 (default, Dec 30 2018, 08:55:50)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.__version__
'1.15.4'
>>> np.asmatrix(np.ones((3, 3)))
/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/matrixlib/defmatrix.py:68: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
return matrix(data, dtype=dtype, copy=False)
matrix([[1., 1., 1.],
[1., 1., 1.],
[1., 1., 1.]])
For convenience, the message text is:
PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
Hi,
In the eval_ineq_constraints()
method in Keane.py, it seems to me that the code
vec = np.zeros((x.shape[0], 2))
vec[:, 0] = 0.75 - np.prod(x)
vec[:, 1] = np.sum(x) - 7.5 * self.dim
should be
vec = np.zeros((a.shape[0], 2))
vec[:, 0] = 0.75 - np.prod(a, axis=1)
vec[:, 1] = np.sum(a, axis=1) - 7.5 * self.dim
I am basing my understanding on the documentation on the expected form of eval_ineq_constraints()
method explained here.
When a function evaluation doesn't complete (error in the external objective function or some other issues), I raise ValueError. However, after raising ValueError pySOT doesn't continue. I assume is getting stuck in some infinite loop, since the process continue to use the CPU 100%. I left it for several hours and didn't change.
To reproduce follow these steps:
virtualenv venv_test
. ./venv_test/bin/activate
pip install numpy
pip install inspyred
git clone [email protected]:dme65/pySOT.git
cd pySOT/
# edit the Ackley class so it raises ValueError
vim pySOT/test/Ackley.py
python setup.py install
# edit test_simple.py so it uses the new Ackley
vim pySOT/test/test_simple.py
cd pySOT/test/
python test_simple.py
Output:
(venv_test)ilija@deep03:~/for_test/pySOT/pySOT/test$ python test_simple.py
Number of threads: 1
Maximum number of evaluations: 1000
Search strategy: CandidateDYCORS
Experimental design: Latin Hypercube
Ensemble surrogates: Cubic RBF, domain scaled to unit box
10-dimensional Ackley function
Global optimum: f(0,0,...,0) = 0
[-12.61363636 -4.65909091 6.47727273 -9.43181818 9.65909091
-12.61363636 8.06818182 -4.65909091 -6.25 16.02272727]
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "build/bdist.linux-x86_64/egg/poap/controller.py", line 426, in run
self.handle_eval(record)
File "build/bdist.linux-x86_64/egg/poap/controller.py", line 453, in handle_eval
self.finish_success(record, self.objective(*record.params))
File "/home/ilija/for_test/pySOT/pySOT/test/Ackley.py", line 19, in objfunction
raise ValueError('Dimension mismatch')
ValueError: Dimension mismatch
I have attached the modified Ackley.py
and test_simple.py
.
Hi,
Is it possible to use pySOT
with expensive blackbox constraints?
For example, if the objfunction()
method in the optimisation problem class can be written to use an external piece of simulation software, then how might this be applied for the eval_ineq_constraints()
method?
In my case, I would be interpreting simulation results from an external program to provide objective function values and constraint function values to pySOT
. But the eval_ineq_constraints()
method seems to accept a numpy
array for a number of different points all at once, unlike the objfunction()
method which accepts one point at a time to be evaluated (if I understand the ReadTheDocs documentation properly). Also, it seems to me that it is possible that objfunction()
and eval_ineq_constraints()
are not necessarily called as a pair - objfunction(x)
could be called multiple times for different x
before a single call to eval_ineq_constraints()
might be performed which evaluates the constraint functions for multiple points during that single call to eval_ineq_constraints()
.
I think it is possible for a user to write their objfunction()
and eval_ineq_constraints()
in a clever way to be able to make efficient use of each call to the external simulation program, e.g. storing both objective and constraint function values from an external simulation run for a point even, if invoked by only one of objfunction()
or eval_ineq_constraints()
; hence the stored values can be checked by future objfunction()
or eval_ineq_constraints()
calls later in the optimisation to avoid repeating external simulations if we already have the information stored by an earlier call.
In addition, I have another related question: if there are to be 500 objective function evaluations in an optimisation, then there would also be 500 evaluations for each constraint, where the objective and constraint functions are evaluated on the exact same set of points? I am not familiar enough with either optimisation in general or pySOT
in particular to know for sure.
Many thanks.
I'm using pySOT as the example 11.6 external C++ objective function as shown here https://github.com/dme65/pySOT/blob/444070e31887525ed418166d9107b6a355353fc9/docs/pySOT.pdf.
When I set the number of evaluations to 2 for example, it will repeat those same 2 over and over again. It would never finish.
If needed I can share the code privately.
Full log:
(.venv)ilija@deep03:~/h$ python pySOT_runner.py 1 2
Number of threads: 1
Maximum number of evaluations: 2
Search strategy: Candidate DyCORS
Experimental design: Latin Hypercube
Surrogate: Cubic RBF
{
epoch : 17
showProgress : false
seed : 139
hiddenNodes : 130
experimentId : 1
threads : 16
momentum : 0.8071425
deviceId : 1
batchSize : 128
learnRate : 0.05075
plot : false
save : "logs"
mean : 0.002500075
std : 0.00453571428571
learnRateDecay : 0
}
0.1937
{
epoch : 19
showProgress : false
seed : 139
hiddenNodes : 173
experimentId : 1
threads : 16
momentum : 0.8214275
deviceId : 1
batchSize : 128
learnRate : 0.0223214285714
plot : false
save : "logs"
mean : 0.000357239285714
std : 0.0893928571429
learnRateDecay : 0
}
0.6733
{
epoch : 17
showProgress : false
seed : 139
hiddenNodes : 130
experimentId : 1
threads : 16
momentum : 0.8071425
deviceId : 1
batchSize : 128
learnRate : 0.05075
plot : false
save : "logs"
mean : 0.002500075
std : 0.00453571428571
learnRateDecay : 0
}
0.1937
{
epoch : 19
showProgress : false
seed : 139
hiddenNodes : 173
experimentId : 1
threads : 16
momentum : 0.8214275
deviceId : 1
batchSize : 128
learnRate : 0.0223214285714
plot : false
save : "logs"
mean : 0.000357239285714
std : 0.0893928571429
learnRateDecay : 0
}
0.6733
And it will repeat this forever.
These are the parameters to optimise
{
epoch : 19
showProgress : false
seed : 139
hiddenNodes : 173
experimentId : 1
threads : 16
momentum : 0.8214275
deviceId : 1
batchSize : 128
learnRate : 0.0223214285714
plot : false
save : "logs"
mean : 0.000357239285714
std : 0.0893928571429
learnRateDecay : 0
}
and 0.6733
is the function evaluation result. You can see it repeats the same two function evaluations, i.e. it doesn't try with different set of parameters.
When installing pySOT
with pip
, I think that examples
and tests
were supposed to be inside the pySOT
package, but they are installed in a directory outside of it.
Hi,
since I'm not very experienced in surrogate-based optimization I'd like to ask whether there is in pySOT some possibility to use points with already known objective function values before starting optimization itself? Is it possible to do it by own implementation of experimental design? But what are the most important properties which that experimental of design has to fulfil? (I’ve just read some information in online documentation.)
Note: My question is motivated by my optimization task. I need to optimize SW (image processing) module where SW efficiency (objective function value) is possible to get by setting parameters and running time-consuming simulation only. But I have already had some parameters settings, which could be used as "initial" point. Is it good idea? Moreover, there are some other known points as a result of previous testing which may be possibly useful as well (?).
Radek
Thanks
Just saw that my new docker image pulled your newest version. Normally i used to wrap the RBFInterpolant into the SurrogateUnitBox(model=RBFInterpolant()). Now SurrogateUnitBox doesn't exist anymore. Before my models where super bad without. Did you fix it now and it works without?
Thanks for your help
Cheers
Fabi
In gp.py
, the predict()
method returns an array of size (num_pts,)
. This does not match the returned size of the predict_std()
method, which instead returns an array of size (num_pts,1)
.
This causes issues in ei_merit.py
, in the ei_merit()
function. When mu
and sig
are assigned different sizes on line 23, gamma
(and subsequently beta
then ei
) will become a square array of size (num_pts, num_pts)
. This causes an IndexError
exception on line 31, when the incorrectly-sized ei
is indexed by the correctly-sized dmerit
.
Potential solution:
In gp.py
, in the predict()
method, change line 71 to more closely match the form of line 85. Suggestion below:
return self._mu + self._sigma * np.expand_dims(self.model.predict(xx), axis=1)
Note: this issue appeared suddenly for me, without any updates to pySot itself, so it is possible that either:
In surrogate.py
, in the add_points()
method, an AttributeError
exception is raised by calling fx.ndim
(line 64) when fx
is an integer.
Potential solutions:
eval
method must return a float. The comments inside add_points()
already reflect this, but the documentation on readthedocs.io does not.Hi,
I am trying to understand the role of setting the extra
and extra_vals
parameters in something like SRBFStrategy()
. It seems like if I append the controller's fevals
list with appropriately populated EvalRecord
objects (as shown in this example), the controller is able to leverage this prior information. What is purpose of also setting the extra
, extra_vals
parameters in SRBFStrategy()
?
Regards
Currently the parameter configuration is being set in this way:
self.xlow = -15 * np.ones(dim)
self.xup = 20 * np.ones(dim)
self.integer = []
self.continuous = np.arange(0, dim)
This is very cumbersome interface, especially when you have 15 parameters, each with different ranges, and are of different type. It's very easy to make a mistake.
Further if the type of parameter is left unassigned, there is no error or warning. Example
self.integer = [0, 1, 2]
self.continuous = [4, 5]
Here the fourth parameter is left without a type (integer or continuous), but there is no error or warning produced, and the algorithm just considers it as continuous.
Also, it would be useful if we can assign a name to a parameter.
I suggest to adopt some JSON configuration file, similar to what other optimization algorithms use. Example:
{
parameters:[
{
name: "learning_rate",
low: 10,
high: 20,
type: "Integer"
},
...
]
}
Most importantly produce an error or at least a warning if not all of the configuration properties are set.
Thanks
pyDOE hasn't been released for more than a year and it seems to be no longer actively maintained. I suggest changing the dependency so that users of pySOT may use the framework without any difficulties. As to the date, I am able to use the pySOT.experimental_design.TwoFactorial methods only if I change the pyDOE's file manually, which turns out to make the deployment of any applications more difficult.
pySOT/pySOT/experimental_design.py
Line 14 in c3f2fce
Hello,
Could you please give me an example that prints out "the uncertainty of the surrogate model at points x" as said in the online manual?
predict_std(x): Evaluates the uncertainty of the surrogate model at points x
I am using your code to calibrate a highly non-linear groundwater model with a high number of model parameters (large than 40). The code worked perfectly and saved me a lot of time. I looking forward to see if I can use the code for uncertainty analysis related to estimated model parameters. I am using pysot with DYCORStrategy.
Thanks,
Hai.
It would be very useful if it's possible to save the state of the program after every function evaluation. One function evaluation can take several hours, so server crash after 200 or something function evaluation is very painful 😅
pySOT performs significantly worse on 200-dimensional problems than was reported for the DYCORS method in "Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization", Regis and Shoemaker.
pySOT automatically restarted before version 0.2.0, but this option was removed to allow users to look at the results before deciding to restart. We should add back an option for the strategies to restart if it's desired by the user to do all function evaluations without looking at any intermediate results.
Problem 1: cannot install inspyred
because of missing README.md
file.
Error log:
Processing dependencies for pySOT==0.1.18
Searching for inspyred
Reading https://pypi.python.org/simple/inspyred/
Best match: inspyred 1.0.1
Downloading https://pypi.python.org/packages/source/i/inspyred/inspyred-1.0.1.tar.gz#md5=ce51b599f5682ec526c97412f3ebaf45
Processing inspyred-1.0.1.tar.gz
Writing /tmp/easy_install-3cWNNt/inspyred-1.0.1/setup.cfg
Running inspyred-1.0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-3cWNNt/inspyred-1.0.1/egg-dist-tmp-_n29fg
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.14.tar.gz
Extracting in /tmp/easy_install-3cWNNt/inspyred-1.0.1/temp/tmpD3lh4S
Now working in /tmp/easy_install-3cWNNt/inspyred-1.0.1/temp/tmpD3lh4S/distribute-0.6.14
Building a Distribute egg in /tmp/easy_install-3cWNNt/inspyred-1.0.1
/tmp/easy_install-3cWNNt/inspyred-1.0.1/distribute-0.6.14-py2.7.egg
error: [Errno 2] No such file or directory: '/tmp/easy_install-3cWNNt/inspyred-1.0.1/README.md'
After manually installing inspyred
and removing the need for README.md
by editing inspyred's
setup.cfg
I have this problem:
Problem 2:
Processing dependencies for pySOT==0.1.18
Traceback (most recent call last):
File "setup.py", line 15, in <module>
'Programming Language :: Python :: 2.7',
File "/home/ilija/anaconda2/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/home/ilija/anaconda2/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/home/ilija/anaconda2/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/install.py", line 73, in run
self.do_egg_install()
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/install.py", line 101, in do_egg_install
cmd.run()
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/easy_install.py", line 342, in run
self.easy_install(spec, not self.no_deps)
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/easy_install.py", line 562, in easy_install
return self.install_item(None, spec, tmpdir, deps, True)
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/easy_install.py", line 614, in install_item
self.process_distribution(spec, dist, deps)
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/easy_install.py", line 666, in process_distribution
[requirement], self.local_index, self.easy_install
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/pkg_resources.py", line 557, in resolve
requirements.extend(dist.requires(req.extras)[::-1])
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/pkg_resources.py", line 2172, in requires
dm = self._dep_map
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/pkg_resources.py", line 2164, in _dep_map
for extra,reqs in split_sections(self._get_metadata(name)):
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/pkg_resources.py", line 2628, in split_sections
for line in yield_lines(s):
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/pkg_resources.py", line 1854, in yield_lines
for ss in strs:
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/pkg_resources.py", line 2186, in _get_metadata
for line in self.get_metadata_lines(name):
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/pkg_resources.py", line 1174, in get_metadata_lines
return yield_lines(self.get_metadata(name))
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/pkg_resources.py", line 1166, in get_metadata
return self._get(self._fn(self.egg_info,name))
File "/home/ilija/anaconda2/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/pkg_resources.py", line 1231, in _get
return self.loader.get_data(path)
zipimport.ZipImportError: bad local file header in /home/ilija/anaconda2/lib/python2.7/site-packages/pySOT-0.1.18-py2.7.egg
So I gave up, and installed via pip install pySOT
Hey guys, for the ub and lb you declare it as a function. But when you pass it later to unit box you declare it in unit box function docu as a np.array. I think the func should just be replaced by np.array
:type lb: function
:param ub: Upper variable bounds, of size 1 x dim
:type ub: function
Cheers :)
Hi all, is this available in pySOT already? From searching in the documentation and previous issues, I could only find two stopping criteria, one based on the maximum number of function evaluations and another on maximum run time.
I would like to stop the search when both my objective and decision variables (norm) do not change (within a tolerance) from one iteration to the next. In my case, sometimes the solver converges within 10 evaluations, other times it takes 20 evaluations, and in the former I would like to obtain an answer after those 10 evaluations since the remaining 10 evaluations may take up to a day to complete.
Thank you.
We should implement a strategy that is tailored for mixed-integer problems, such as SO-MI (A surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems, Müller, Juliane; Shoemaker, Christine; Piche, Robert)
The file requirements.txt
is missing. It is referenced here:
https://github.com/dme65/pySOT/blob/master/docs/pySOT.tex#L383
In ei_merit.py
, in the ei_merit()
method, the sizes of the parameters passed do not match the comments.
My setup:
To reproduce: put a breakpoint on line 23 in ei_merit()
and use the debug console to observe the following:
np.shape(X)
(100, 6)
np.shape(fX)
(6, 1)
np.shape(XX)
(18, 6)
Two semi-separate issues:
The comment on line 15 states that parameter XX
should have shape m x 1
. It should actually by m x dim
since XX
represents inputs to the objective function, which have size equal to the dimension of the problem.
I think this should be an easy fix, assuming that I diagnosed the issue correctly.
The comments on lines 13 and 15 collectively state that the dimensions on axis 0 of fX
and XX
should match (and be equal to m
). While this would make sense to me, I have not observed that in practice (see console output above). I believe this discrepancy can be traced back to line 33 of ei_ga.py
, where the pending evaluations are appended to the previously evaluated points before later passing them to ei_merit
on line 41. This makes the array of previously evaluated points larger than the array of values at previously evaluated points, by an amount equal to the number of pending evaluations.
I am not sure if I correctly diagnosed the root cause of this issue. While the comment is definitely wrong in the current state, I do not know whether the comment or the code meets the intent of the design. As far as I can tell, the code in its current state does not cause any problems with effectiveness or reliability.
I am using pySOT with EIStrategy and GPRegressor surrogate model for a 6-dimensional optimization problem (all axes are continuous). I am using 10 worker threads. I have Python 3.11.1 and Windows 10.
In recent runs, I have observed that after a little more than 1000 data points, generating new points starts to slow WAY down. Before this occurs, most time is spent evaluating the objective function, with worker threads being assigned a new task within seconds (or ms even) of finishing the previous evaluation. However, at a little over 1000 data points, I noticed that almost all the time is spent waiting for new assignments.
This is confirmed by computing resource allocation. Prior to 1k, the evaluations called by worker threads take up all available CPU. After 1k, Python takes ~30% and worker threads rarely take any at all. In fact, only one worker thread is ever actually evaluating the objective at once because it gets done evaluating the objective before the next point has been generated.
This leads me to suspect that the process of generating new points slows WAY down around 1k points.
Edit: this SE article seems relevant. https://stats.stackexchange.com/questions/326446/gaussian-process-regression-for-large-datasets
In surrogate_strategy.py I believe there is supposed to be a parenthesis on line 163 otherwise it never catches the error of having less evaluation points than the minimum. Currently in the example problems I can put any value for max_evals when calling strategy and the error won't be thrown.
The module ez_setup is also required (https://github.com/dme65/pySOT/blob/master/setup.py#L13)
Hi, I am trying the examples provided and the MPI example gets stuck, not error, just not advancing nor calling the objective function. I have tried the same code under windows 10 (after installing MPI from microsoft) and under Ubuntu 18.
I wonder is this is a bug, of if I am missing something.
The code is this:
"""
.. module:: mpiexample_simple_mpi
:synopsis: Simple MPI example
.. moduleauthor:: David Eriksson <[email protected]>
"""
from pySOT.experimental_design import SymmetricLatinHypercube
from pySOT.strategy import SRBFStrategy
from pySOT.surrogate import RBFInterpolant, CubicKernel, LinearTail
from pySOT.optimization_problems import OptimizationProblem
from poap.mpiserve import MPIController, MPISimpleWorker
import numpy as np
import os.path
import logging
# Try to import mpi4py
try:
from mpi4py import MPI
except Exception as err:
print("ERROR: You need mpi4py to use the POAP MPI controller.")
exit()
class Ackley(OptimizationProblem):
"""
Ackley function
.. math::
f(x_1,\\ldots,x_n) = -20\\exp\\left( -0.2 \\sqrt{\\frac{1}{n} \
\\sum_{j=1}^n x_j^2} \\right) -\\exp \\left( \\frac{1}{n} \
\\sum{j=1}^n \\cos(2 \\pi x_j) \\right) + 20 - e
subject to
.. math::
-15 \\leq x_i \\leq 20
Global optimum: :math:`f(0,0,...,0)=0`
:ivar dim: Number of dimensions
:ivar lb: Lower variable bounds
:ivar ub: Upper variable bounds
:ivar int_var: Integer variables
:ivar cont_var: Continuous variables
:ivar min: Global minimum value
:ivar minimum: Global minimizer
:ivar info: String with problem info
"""
def __init__(self, dim=10):
self.dim = dim
self.min = 0
self.minimum = np.zeros(dim)
self.lb = -15 * np.ones(dim)
self.ub = 20 * np.ones(dim)
self.int_var = np.array([])
self.cont_var = np.arange(0, dim)
self.info = str(dim) + "-dimensional Ackley function \n" + "Global optimum: f(0,0,...,0) = 0"
def eval(self, x):
"""
Evaluate the Ackley function at x
:param x: Data point
:type x: numpy.array
:return: Value at x
:rtype: float
"""
print('eval!')
self.__check_input__(x)
d = float(self.dim)
return -20.0 * np.exp(-0.2*np.sqrt(np.sum(x**2) / d)) - np.exp(np.sum(np.cos(2.0*np.pi*x)) / d) + 20 + np.exp(1)
def main_worker(objfunction):
MPISimpleWorker(objfunction).run()
def main_master(opt_prob, num_workers):
if not os.path.exists("./logfiles"):
os.makedirs("logfiles")
if os.path.exists("./logfiles/mpiexample_mpi.log"):
os.remove("./logfiles/mpiexample_mpi.log")
logging.basicConfig(filename="./logfiles/mpiexample_mpi.log",
level=logging.INFO)
max_evals = 500
rbf = RBFInterpolant(dim=opt_prob.dim,
kernel=CubicKernel(),
tail=LinearTail(opt_prob.dim))
slhd = SymmetricLatinHypercube(dim=opt_prob.dim,
num_pts=2*(opt_prob.dim+1))
# Create a strategy and a controller
strategy = SRBFStrategy(max_evals=max_evals,
opt_prob=opt_prob,
exp_design=slhd,
surrogate=rbf,
asynchronous=True,
batch_size=num_workers)
controller = MPIController(strategy)
print("Number of workers: {}".format(num_workers))
print("Maximum number of evaluations: {}".format(max_evals))
print("Strategy: {}".format(controller.strategy.__class__.__name__))
print("Experimental design: {}".format(slhd.__class__.__name__))
print("Surrogate: {}".format(rbf.__class__.__name__))
print('Running...')
result = controller.run()
print('Best value found: {0}'.format(result.value))
print('Best solution found: {0}\n'.format(np.array_str(result.params[0],
max_line_width=np.inf,
precision=5,
suppress_small=True)))
def mpi_example_simple():
# Optimization problem
ackley = Ackley(dim=10)
# Extract the rank
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
nprocs = comm.Get_size()
if rank == 0:
main_master(ackley, nprocs)
else:
main_worker(ackley.eval)
if __name__ == '__main__':
mpi_example_simple()
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.