Adapting A2C and deep policy gradient methods to Discrete envs about mushroom-rl HOT 4 CLOSED

Hi,
Currently, we don't support actor-critic methods with finite actions.
However, I think it would be quite easy to make it work for A2C, given the simplicity of the algorithm.
Indeed, you are trying to use a gaussian policy in a discrete action environment. What you should do, instead, is implementing a discrete policy (e.g. boltzmann policy) extending the torch policy interface. it should be quite easy btw.

the only problematic part is to remember to cast back to long the action that is converted to float tensor inside the _loss method (or change the line in the a2c code)

from mushroom-rl.

Hi, thanks for the response! Do you know if there is a deep RL method in your library that supports finite actions? Apart from DQN? I will make the necessary changes for A2C if there aren't any available.

from mushroom-rl.

Mostly is missing the policy. After that you should be almost done.

All variants of dqn (double, averaged, categorical) support finite actions.

no actor-critic method currently supports finite actions.

You could also try fitted q iteration with deep networks, even if this algorithm works better with extra trees.

from mushroom-rl.

We just pushed the BoltzmannTorchPolicy in the dev branch.
see commit 4d20e68
Also, there is an example of usage of this policy with a2c here

This should fix your issue. If not, feel free to open another issue/bug report.

from mushroom-rl.