igitugraz / l2l Goto Github PK
View Code? Open in Web Editor NEWLearning to Learn: Gradient-free Optimization framework
Home Page: https://igitugraz.github.io/L2L/
License: GNU General Public License v3.0
Learning to Learn: Gradient-free Optimization framework
Home Page: https://igitugraz.github.io/L2L/
License: GNU General Public License v3.0
It seems like bin/ltl-fun-gs.py is using the old FuntionOptimizee interface.
Including RMSProp
Implementation available in scikit-learn
The fact that none of the optimizers have any capabilities for seeding is problematic in the least. I think the following two features will be required for all optimizers that require random number generation,
Hi! In this portion of the optimizer, what I understand is that you first check if there are repeated individuals, and then you want to mutate those repeated
L2L/l2l/optimizers/evolution/optimizer.py
Lines 173 to 180 in 5e1695a
But in line 176, I think the iteration should start with i+1
instead of i
. Otherwise , since your second loop starts with the same individial as the first one, you would be mutating all individuals 80% probability. Furthermore, I believe that, in line 175, offspring should iterate on [:-1]
.
The postprocess function is called recursively and the stack is only unwinded after finishing all computations. This results in heavy memory consumption when using local variables within the postprocess function.
The following lines are creating multiple directories at the same level as the parent
Lines 56 to 65 in ef273cb
Paths should be generated using os.path.join(parent, child). For example,
xml_path = os.path.join(self.path, "jube_xml")
if not os.path.exists(xml_path):
os.makedirs(xml_path)
The real issue is presented further down in the source code. This seems to be the same as the initial setting
Line 76 in bb305d4
Lines 130 to 131 in bb305d4
The main issue is that, by manually generating the paths, we lose compatibility between OS-s and risk misplacing files.
We want the step size to be a hyper-parameter of the optimizee rather than an optimizer hyper-parameter, since each optimizee has different requirements, and may even have different required step sizes for each dimension.
One of the solutions suggested by @guillaumeBellec would be to allow the current bounding function to also scale individual so that all optimizers only have to generate individuals between 0 and 1. The optimizee knows the required scaling along with the bounding and returns an appropriately scaled and bounded individual.
We have to think about how this would affect the optimizers we have, since it would change the error landscape they see. Specifically, how would the cross-entropy optimizer have to be modified to be able to do this? (i.e. how would it compensate for the scaling?).
@maharjun @bohnstingl @franzscherr Any thoughts about this?
Other possible ways to achieve optimizer independent step-size are also welcome.
I cannot seem to think of a way to implement smoothing for gaussian mixture models that makes any sense. This is because in the current implementation, there is absolutely no logic to decide which component gets linearly combined with which. This leads to pretty arbitrary smoothing I would think. Is there any way to do this in a manner that makes sense?
A dummy Optimizer that does a grid search over a given range of parameters.
I'm using the simulated annealing optimizer, with the following parameters:
parameters = SimulatedAnnealingParameters(n_parallel_runs=1, noisy_step=.1, temp_decay=0.95, n_iteration=150, stop_criterion=np.Inf, seed=np.random.randint(1e5), cooling_schedule=AvailableCoolingSchedules.DEFAULT)
optimizer = SimulatedAnnealingOptimizer(traj, optimizee_create_individual=optimizee.create_individual, optimizee_fitness_weights=(1,), parameters=parameters, optimizee_bounding_func=optimizee.bounding_func)
I'm returning from my optimizee a fitness between 0 and 1 (1 being the best), but still, the optimizer is trying to minimize that value.
Is that normal?
Ideas from David's work.
Especially in noisy environments it could be useful that the parameters chosen by the optimizer are evaluated multiple times to better estimate the mean and the variance of the optimizees fitness.
See work from Kroese and Rubenstein
The plan is to generate a markdown snippet that contains the following information: name of Function, parameters, name of optimizer, parameters, number of iterations, optima found, actual optima (if known), time of run (approx), current git commit id (no uncommited changes), date!
The markdown snippet is generated using a Jinja2 template.
Git commit id is included using GitPython
The class has format:
class Recorder:
def start():
# Check no uncommitted changes (throw error if not)
def end(...args...):
# Generate markdown file with the above fields in the current directory
# Optionally, the markdown file contains the cost for each iteration (and possibly plot)
Usage:
recorder = Recorder()
recorder.start() # Start before everything else
# Do your experiment
recorder.end() # Last line of code
The user uploads the generated markdown file wherever -- GitHub wiki, their own repository etc.
How do we construct such functions? (references)
Seeing as how pypet leaking memory isn't enough, we also have SCOOP filling in for it. I have currently plugged the leak on my own fork and will put up the relevant pull request with the changes shortly. Until that pull request is merged into scoop master, we will go with my fork.
SCOOP BUG: SCOOP basically does not read out of the pipes to which the remote stdout is attached. This means that if the remote process prints more than 64 kb of data to stdout, that subprocess will hang and your simulations will never complete. Experienced this while trying to run the Ison Memory Simulation experiment and by observing the SCOOP code.
I have three questions regarding the benchmark evaluation:
The commit ID is kind of misleading, since one has to commit at least once to set the path for local execution. The problem is that the commit is only locally and cannot be pushed to the master directly.
How can we handle this?
The parameters for the individuals used to get the best performance are not tracked.
This might be also an interesting information?
The field "actual optima" seems to be None at every execution.
Am I doing something wrong or is this not implemented yet?
Thanks for your help!
Instead of using the style-checker run-style-check.sh
I would like to recommend to use pep8 speaks instead. One can configure it in the same fashion as the checker script and it can run e.g. only on the code of the actual pull request. Right now the CI always breaks because of all pep8 issues from all files. Is this a volitional behavior?
Should it be a weighted sum (as it is currently) or a lexicographic ordering?
Any tips on how can this be solved? And where? Since LTL-SPORE is based on vanilla pattern matching experiment from spore-nest-module, would it be easier to adapt the optimizer? On the other hand optimizers should be decoupled from optimizees.
Console output:
$ python3 -m scoop bin/ltl-pm-gd.py
[2017-06-01 13:05:02,949] launcher INFO SCOOP 0.7 1.1 on linux using Python 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609], API: 1013
[2017-06-01 13:05:02,949] launcher INFO Deploying 4 worker(s) over 1 host(s).
[2017-06-01 13:05:02,949] launcher INFO Worker distribution:
[2017-06-01 13:05:02,950] launcher INFO 127.0.0.1: 3 + origin
All output can be found in file /home/dsalaj/Documents/CI_project/LTL-SPORE/results/LTL-PatternMatching-GD/run-no-test/results/output.log
Change the values in logging.yaml to control log level and destination
e.g. change the handler to console for the loggers you're interesting in to get output to stdout
MainProcess STDOUT INFO Established redirection of `stdout`.
MainProcess snn_utils.comm.zmq INFO Subscribing to tcp://127.0.0.1:8001
Traceback (most recent call last):
File "./zmq_proxy_node_py3.py", line 7, in <module>
ZMQProxyNode().main()
File "/home/dsalaj/Documents/CI_project/LTL-SPORE/spore/snn_utils/comm/music/node/__init__.py", line 75, in main
self._setup(self._music_setup)
File "/home/dsalaj/Documents/CI_project/LTL-SPORE/spore/zmq_proxy_node.py", line 57, in _setup
self._server = Publisher(music_zmq_proxy_config['communication']['port'])
File "/home/dsalaj/Documents/CI_project/LTL-SPORE/spore/snn_utils/comm/zmq/__init__.py", line 29, in __init__
self._sock.bind(address)
File "zmq/backend/cython/socket.pyx", line 495, in zmq.backend.cython.socket.Socket.bind (zmq/backend/cython/socket.c:5653)
File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc (zmq/backend/cython/socket.c:10014)
zmq.error.ZMQError: Address already in use
[ubuntu-x230:16282] *** An error occurred in MPI_Group_free
[ubuntu-x230:16282] *** on communicator MPI_COMM_WORLD
[ubuntu-x230:16282] *** MPI_ERR_GROUP: invalid group
[ubuntu-x230:16282] *** MPI_ERRORS_ARE_FATAL: your MPI job will now abort
Traceback (most recent call last):
File "./zmq_proxy_node_py3.py", line 7, in <module>
ZMQProxyNode().main()
File "/home/dsalaj/Documents/CI_project/LTL-SPORE/spore/snn_utils/comm/music/node/__init__.py", line 75, in main
self._setup(self._music_setup)
File "/home/dsalaj/Documents/CI_project/LTL-SPORE/spore/zmq_proxy_node.py", line 57, in _setup
self._server = Publisher(music_zmq_proxy_config['communication']['port'])
File "/home/dsalaj/Documents/CI_project/LTL-SPORE/spore/snn_utils/comm/zmq/__init__.py", line 29, in __init__
self._sock.bind(address)
File "zmq/backend/cython/socket.pyx", line 495, in zmq.backend.cython.socket.Socket.bind (zmq/backend/cython/socket.c:5653)
File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc (zmq/backend/cython/socket.c:10014)
zmq.error.ZMQError: Address already in use
Traceback (most recent call last):
File "./zmq_proxy_node_py3.py", line 7, in <module>
ZMQProxyNode().main()
File "/home/dsalaj/Documents/CI_project/LTL-SPORE/spore/snn_utils/comm/music/node/__init__.py", line 75, in main
self._setup(self._music_setup)
File "/home/dsalaj/Documents/CI_project/LTL-SPORE/spore/zmq_proxy_node.py", line 57, in _setup
self._server = Publisher(music_zmq_proxy_config['communication']['port'])
File "/home/dsalaj/Documents/CI_project/LTL-SPORE/spore/snn_utils/comm/zmq/__init__.py", line 29, in __init__
self._sock.bind(address)
File "zmq/backend/cython/socket.pyx", line 495, in zmq.backend.cython.socket.Socket.bind (zmq/backend/cython/socket.c:5653)
File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc (zmq/backend/cython/socket.c:10014)
zmq.error.ZMQError: Address already in use
--------------------------------------------------------------------------
mpirun has exited due to process rank 2 with PID 16282 on
node ubuntu-x230 exiting improperly. There are two reasons this could occur:
1. this process did not call "init" before exiting, but others in
the job did. This can cause a job to hang indefinitely while it waits
for all processes to call "init". By rule, if one process calls "init",
then ALL processes must call "init" prior to termination.
2. this process called "init", but exited without calling "finalize".
By rule, all processes that call "init" MUST call "finalize" prior to
exiting or it will be considered an "abnormal termination"
This may have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).
--------------------------------------------------------------------------
[ubuntu-x230:16291] *** An error occurred in MPI_Group_free
[ubuntu-x230:16291] *** on communicator MPI_COMM_WORLD
[ubuntu-x230:16291] *** MPI_ERR_GROUP: invalid group
[ubuntu-x230:16291] *** MPI_ERRORS_ARE_FATAL: your MPI job will now abort
[ubuntu-x230:16287] *** An error occurred in MPI_Group_free
[ubuntu-x230:16287] *** on communicator MPI_COMM_WORLD
[ubuntu-x230:16287] *** MPI_ERR_GROUP: invalid group
[ubuntu-x230:16287] *** MPI_ERRORS_ARE_FATAL: your MPI job will now abort
--------------------------------------------------------------------------
mpirun has exited due to process rank 2 with PID 16287 on
node ubuntu-x230 exiting improperly. There are two reasons this could occur:
1. this process did not call "init" before exiting, but others in
the job did. This can cause a job to hang indefinitely while it waits
for all processes to call "init". By rule, if one process calls "init",
then ALL processes must call "init" prior to termination.
2. this process called "init", but exited without calling "finalize".
By rule, all processes that call "init" MUST call "finalize" prior to
exiting or it will be considered an "abnormal termination"
This may have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun has exited due to process rank 2 with PID 16291 on
node ubuntu-x230 exiting improperly. There are two reasons this could occur:
1. this process did not call "init" before exiting, but others in
the job did. This can cause a job to hang indefinitely while it waits
for all processes to call "init". By rule, if one process calls "init",
then ALL processes must call "init" prior to termination.
2. this process called "init", but exited without calling "finalize".
By rule, all processes that call "init" MUST call "finalize" prior to
exiting or it will be considered an "abnormal termination"
This may have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).
--------------------------------------------------------------------------
as reported by the optimizee and not the fitness multiplied by the fitness weights (which should only be used internally by the optimizer).
From the appendix of:
Onbaşoğlu, Esin, and Linet Özdamar. “Parallel Simulated Annealing Algorithms in Global Optimization.” Journal of Global Optimization 19, no. 1 (January 1, 2001): 27–50. https://dx.doi.org/10.1023/A:1008350810199.
It appears that even after the latest patch scoop is still losing futures. This things appears to be meant as some sort of insurance against a worker failing midway, however, due to issues it doesn't seem to work correctly. I have some idea of what's wrong and will get around to fixing it. However in the meanwhile, I would suggest the following workaround
go to the scoop code (to find out the directory, type import scoop; scoop.__file__
in an IPython console) to the file _types.py
to line 389 and comment out self.askForPreviousFutures()
. SCOOP will stop losing futures. Although the robustness to a lost worker will be gone, this should be fine considering we don't really lose workers ever.
E.g. strings, or just a list of possible values.
Currently, it seems like only CE has default parameters.
So while doing the benchmark evaluations today, with grid search optimizer for 10D
cases(you can change function_id = 1 in bin/ltl-fun-gs.py) this error was thrown:
Traceback (most recent call last):
File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/sinisa/.local/lib/python3.4/site-packages/scoop/bootstrap/__main__.py", line 302, in <module>
b.main()
File "/home/sinisa/.local/lib/python3.4/site-packages/scoop/bootstrap/__main__.py", line 92, in main
self.run()
File "/home/sinisa/.local/lib/python3.4/site-packages/scoop/bootstrap/__main__.py", line 290, in run
futures_startup()
File "/home/sinisa/.local/lib/python3.4/site-packages/scoop/bootstrap/__main__.py", line 271, in futures_startup
run_name="__main__"
File "/home/sinisa/.local/lib/python3.4/site-packages/scoop/futures.py", line 64, in _startup
result = _controller.switch(rootFuture, *args, **kargs)
File "/home/sinisa/.local/lib/python3.4/site-packages/scoop/_control.py", line 253, in runController
raise future.exceptionValue
File "/home/sinisa/.local/lib/python3.4/site-packages/scoop/_control.py", line 127, in runFuture
future.resultValue = future.callable(*future.args, **future.kargs)
File "/usr/lib/python3.4/runpy.py", line 240, in run_path
pkg_name=pkg_name, script_name=fname)
File "/usr/lib/python3.4/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "bin/ltl-fun-gs.py", line 99, in <module>
main()
File "bin/ltl-fun-gs.py", line 73, in main
parameters=parameters)
File "/home/sinisa/Uni/Project_CI/LTL/ltl/optimizers/gridsearch/optimizer.py", line 97, in __init__
curr_param_list = np.meshgrid(*([curr_param_list] * param_length), indexing='ij')
File "/home/sinisa/.local/lib/python3.4/site-packages/numpy/lib/function_base.py", line 4532, in meshgrid
mult_fact = np.ones(shape, dtype=int)
File "/home/sinisa/.local/lib/python3.4/site-packages/numpy/core/numeric.py", line 192, in ones
a = empty(shape, dtype, order)
MemoryError
For 2D cases it seems to be working fine though
The formula for the rosenbrock function as given in the mentioned reference and other pages includes a factor of 100 in one of the terms that has been erroneously omitted in the current implementation.
When adding parameter groups in the CE and GS optimizers, they complain that the ParameterDict is not callable.
e.g.
traj.parameters.f_add_parameter_group('grid_spec')
TypeError: 'ParameterDict' object is not callable
Having the value of root_dir in the bin/*.py file and having to edit/un-edit it before each checkin might not be ideal.
One solution, suggested by @dsalaj, would be to have a separate file, that is ignored by git, containing the path string, and the bin/*.py files just read this.
Should be based on the method in published literature
To be consistent with the rest of the optimizers
And report the results from multiple starting points or fixed starting points.
The create_individual function or the spec therefrom should be used instead of having to change this for each optimizee. This breaks interface consistency.
results/benchmark-<id>/<filename>.md
..md
generated in the benchmark should be pasted onto the github wiki in the benchmarks pagePypet has the following issues that are critical to fix
Which is not known in most cases. It should be defined as the delta change in fitness where the optimization is stopped.
The paper [1] proposes alternatives to the CE method. Maybe they are also interesting for LTL purposes?
Since there are papers [1][2] that claim it's the bees knees.
[1] Salimans, Tim, et al. "Evolution Strategies as a Scalable Alternative to Reinforcement Learning." arXiv preprint arXiv:1703.03864 (2017).
[2] Morse, Gregory, and Kenneth O. Stanley. "Simple Evolutionary Optimization Can Rival Stochastic Gradient Descent in Neural Networks." Proceedings of the 2016 on Genetic and Evolutionary Computation Conference. ACM, 2016.
Performance would probably be best individual in that iteration/generation.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.