There are dynamic goal environments. We modify the robotic manipulation environments created by OpenAI (Brockman et al., 2016) for our experiments.

As shown in above figure, we assign certain rules to the goals so that they accordingly move in the environments while an agent is required to control the robotic arm's grippers to reach the goal that moves along a straight line (Dy-Reaching), to reach the goal that moves in a circle (Dy-Circling), or to push a block to the goal that moves along a straight line (Dy-Pushing).

NOTE: The first three tasks need mujoco. The fourth task does not need mujoco but pygame. It is cheaper to only install Dy-Snake.

Our environments depend on openai gym. Please install gym (ver-0.10.9) at first.

cd dygym
python install -e .

cd dygym/test
python test_dyreach.py

Our algorithms depend on openai baselines. Please install baselines (ver-0.1.5) at first.

cd dher
python install -e .

cd dher/ddpg_dher/experiment
python train_dyreach.py

cd dher/dqn_dher/experiment
python train_dysnake.py

NOTE: In Dy-Snake, the first four digits of an observation indicate achieved goals and desired goals. Our implementation of DQN+DHER uses this trick.

This browser does not support PDFs. Please download the PDF to view it: Download PDF.

Please cite our ICLR paper if you use this repository in your publications:

@inproceedings{
fang2019dher,
title={{DHER}: Hindsight Experience Replay for Dynamic Goals},
author={Meng Fang and Cheng Zhou and Bei Shi and Boqing Gong and Jia Xu and Tong Zhang},
booktitle={International Conference on Learning Representations},
year={2019},
url={https://openreview.net/forum?id=Byf5-30qFX},
}

The MIT License