This repo contains code accompanying the paper: Zhi Wang, Chunlin Chen, and Daoyi Dong, "Instance Weighted Incremental Evolution Strategies for Reinforcement Learning in Dynamic Environments", IEEE Transactions on Neural Networks and Learning Systems, 2022. It contains code for running the incremental learning tasks, including 2D navigation, Swimmer, Hopper, and HalfCheetah domains. The basic reinforcement learning algorithms are implemented using natural evolution strategies.

This code requires the following:

• python 3.5+
• pytorch 0.4+
• gym
• MuJoCo license

• For the 2D navigation domain, data is generated from envs/navigation.py
• For the Swimmer/Hopper/HalfCheetah Mujoco domains, the modified Mujoco environments are in envs/mujoco/*

• For example, in Case I of the navigation domain, just run the bash script navi_v1_iwies.sh to get the results of iwies and its ablation methods, also see the usage instructions in the script and main.py; just run the bash script navi_v1_baselines.sh to get the results of the baselines including Robust, Hist, SO-CMA, and ES-MAML, also see the usage instructions in the script and baselines.py
• When getting the results in output/*/*.npy files, plot the results using data_process.py. For example, the results for the navigation domains are as follows:

Case I	Case II	Complex Case

Note that these results are from a single run of the code. You can randomly change the environment to a new one, and record the performance of all tested methods when adapting to the new environment. In our paper, we repeat the process ten times and report the mean and standard error to demonstrate the performance for learning in stochastic dynamic environments. For example, the results for Case I of navigation domain and the swimmer domain are as follows:

navigation_v1	swimmer

Also, the results for other demo scripts are shown in exp/*

To ask questions or report issues, please open an issue on the issues tracker, or email to [email protected].