notnanton / deep-rl-torch Goto Github PK

Add option to have initial CNN layers in every network. This option should be applicable first to the grid representation, then to the pixels.

These layers should be mostly for goal-oriented feature extraction, hence all networks (Q, V, actor) should share them.

I started to write a function that does checks for each hyperparameter how different algorithms react to it. If the performance of one algorithm changes differently from the performance of another algorithm while varying a hyperparameter, then this hyperparameter needs to be tuned separately for each algorithm.

At the moment the Q-learning train and act code still assumes that actions are simply scalar indexes. This needs to be generalized to the case of multiple continuous actions.

To do so a generalization of the whole training approach might be helpful. To achieve this, draw a scheme of the current behavior, a refined version of this, and possible changes that need to be introduced to deal with continuous action spaces (and that needs to be able to be extended to the use of conv. nets).

Put as much code into reusable functions, such as: trainNetwork(network, optimizer, loss) or more general as in trainQnet(Qnet, targetQnet, states, actions, ...)

To act in this environment, continuous actions are necessary. Implement the basic actor-critic methods.

1. CACLA
2. DDPG
3. Off-policy CACLA
4. SPG