sash-a / es_pytorch Goto Github PK

• compute_centered_ranks will push away from the worst value with the same strength that it pulls towards the best value
• create a ranking function that has a stronger pull towards the best and a weaker push from the worst

From testing novelty search seems to be doing as it is programmed, but in practice it performs poorly.

Options:

• compare to openai's method of calculating the novelty metric
• sqrt the distances
• add more behaviors to the initial archive
• compare vector of positions and final position (as a behavioral classification)

• Self adaptive theta as seen in Evolutionary strategies a comprehensive introduction
• Use multiple different theta values for each noise ind. This allows one to search along the trajectory of a specific noise ind.

• Use a temperature param that decreases over time to control the size of the noise - similar to epsilon greedy

• When an individual becomes stuck, reset params to the best performer seen yet

This is already implemented, but the values need to be tuned.

• Should be tested on mujoco to see if it improves performance

simple_example.py

• only stdout reporter
• save in intervals
• objective based
• single individual

• Try both types of recombination as a means to reset the params and explore a different area

Move params in direction of best ever reward. Needs to be done with population

Separate each individuals performance into its own graph.

• graphs for each individual (simply append pop-idx to each graph)
• sub runs on mlflow