CMA-ES inconsistencies about ensmallen HOT 12 OPEN

CMA-ES isn't related to NEAT, but you get to know the optimizer infrastructure.

from ensmallen.

CNE also does the same as CMAES for point 1. Are there any drawbacks to initializing the population by adding matrices generated by arma::randn() to the starting point?

from ensmallen.

Right, no need to restrict the starting point.

from ensmallen.

@zoq For point 2, besides your suggestions, could we also include the initial step size as a parameter? Currently, we experience a problem when we use larger bounds, since the initial step size becomes too large. This causes the CMAES implementation to fail tests when the parameters are changed to include larger bounds (For e.g., [-25,25] for the LogisticRegressionTest). This problem is fixed if we change the step size.

Moreover, I think instead of using one set of bounds for all parameters of a candidate, the user should be able to apply constraints to each parameter separately, since that is more practically useful. Please let me know your thoughts on this

from ensmallen.

Sounds reasonable to me, I think this is already included in #88?

from ensmallen.

No, I wanted to make sure it was alright before including it. I'll make the required changes to #88 soon

from ensmallen.

Sounds good, thanks!

from ensmallen.

Do you want me to wait to release the next version of ensmallen until #88 is merged?

from ensmallen.

I'm not sure whether that question was meant for me, but I should let you know that i might not be able to work on #88 much for the next couple weeks, since I'm going to be busy with exams and stuff.

from ensmallen.

No worries, we can just get #90 merged and then release, assuming that that one is basically ready (I think it is, I'll try and review it tomorrow).

from ensmallen.

Sure :)
Also, about applying box constraints for each parameter, i don't think we currently have any implementation that does that yet. Would it be fine to add a parameter in the constructor for an arma::cube where each slice has the constraints? Or is there some other way you would like to do this?

from ensmallen.

Personally I don't see any issue with an arma::cube for constraints. It might be worth having that as a separate constructor, so that the user can choose to provide two double parameters for the minimum and maximum in each dimension (which is easy to make a hypercube with), or a more complex arma::cube.

from ensmallen.