How to add time limitation in my environment? about tianshou HOT 7 CLOSED

You can use gym.wrapper.
Btw, the limitation of Pendulum-v0 defaults to be 200 frames. So what's your gym version? (0.17.1 for me and works fine, at 200th timestep it returns done=True)

from tianshou.

The reference code:

In [14]: import gym 
    ...: print(gym.__version__) 
    ...: e = gym.make('Pendulum-v0') 
    ...: e.reset() 
    ...: i, d = 0, False 
    ...: while not d: 
    ...:     _, _, d, _ = e.step([0.1]) 
    ...:     i += 1 
    ...: print(i)
0.17.1
200

from tianshou.

I see "done=False“ in sourcecode pendulum.py, you mean "gym.Wrapper" does extra work ( 200th timestep) on Pendulum?

from tianshou.

No, I mean the default gym.make('Pendulum-v0') will not cause this problem. Manually adding gym.wrapper is the last choice.

from tianshou.

I think setting "done=True" after 200 steps is added by "gym.Wrapper".

In your code above, I add "e.unwrapped" after "e = gym.make('Pendulum-v0')", it will not break the while loop.

from tianshou.

In my one Env, using tianshou.ppo to train the agent always warning "'There are already many steps in an episode. You should add a time limitation to your environment!", but Pendulum doesn't warning. At first I think you did something magic in tianshou's code so I open a issue. Sorry for taking your time, Thanks!

from tianshou.

Yes, unwrapped action will remove the time limit added by the gym.wrapper.

from tianshou.