[Bug Report] Maze/AntMaze issues about gymnasium-robotics HOT 4 CLOSED

@alexdavey feel free to make a PR

from gymnasium-robotics.

Your test for sparse rewards, is not correct, since using random actions is very unlikely to get you to the terminal state

Does this also affect the PointEnv?

from gymnasium-robotics.

Thanks for taking a look at this issue.

1. I agree that it is not a robust test in general, it’s primarily just a check to compare pre/post commit (after the suggested changes, it does get a very small but non-zero reward). If you have suggestions for a more robust test I can implement that, but I think the issue is fairly clear from the step method of AntMaze:

and comparing to the PointMaze step:

.compute_terminated() is common to both and resets the goal location if within 0.45.

This issue previously existed for PointMaze. I’m proposing to make the same fix as commit ace181e, but for AntMaze.

2. The PointEnv/PointMaze env’s are not affected by either of these issues, due to the commit referenced above, and because maze_size_scaling = 1 so the missing factor does not change anything.

from gymnasium-robotics.

1. I would say it is worrying that compute_terminated is not a pure function, it should be refactored into 2 functions: compute_terminated (but now it just computes if it is terminated), and update_goal

These changes would requite a new revision ("AntMaze_UMaze-v4")

@rodrigodelazcano

from gymnasium-robotics.