Sticky Actions: Get Actually Executed Action about atari-py HOT 3 CLOSED

There's currently no way to do this. You'd have to implement sticky actions yourself on top of the environment. Out of curiosity why would you want to do this?

from atari-py.

While this may not be a native feature, is there any way I could modify the gym code directly to do something like this that you are aware of?

I am trying to train a model to extract actions from sequences of frames. The offline RL datasets I have been using were generated with repeat_action_probability = 0.25, meaning that 25% of the ground truth action labels in the datasets are garbage. So I wanted to regenerate the exact same data, but with knowledge of the actually executed action. Having the exact same data (frame-by-frame, not just the same protocol) but with the actually executed actions will allow me to use these standard offline RL datasets to benchmark my model.

from atari-py.

So the short answer is yes, you can easily use environment wrappers, check out some of the built-in wrappers here: https://github.com/openai/gym/tree/master/gym/wrappers. You could also just as easily implement this on top of Gym without wrappers, just do the sampling yourself when interacting with the environment.

I still don't fully understand your use case. If you have an offline RL dataset the actions in the dataset are presumably the agent's chosen actions, not the actions the environment will execute, I don't see how 25% of your action labels are garbage? If sticky actions were enabled when you generated the dataset then perhaps 25% of your transitions wouldn't correspond to the ground truth emulation, i.e., S_{t+1} is a stochastic function S_t, A_t due to sticky actions, the only source of stochasticity in the ALE. Do you really want to remove this stochasticity? Generally, it should be assumed that this is part of the environment dynamics.

If you don't want any form of stochasticity you could just generate the dataset with sticky actions disabled?

from atari-py.