Andes environment for OpenAI Gym

• Free software: GPL v3+

Create an empty conda environment with

conda create -n drl-andes python=3.8

The detailed instruction of Andes installation can be found here.

conda config --add channels conda-forge
conda config --set channel_priority flexible

Use conda install instead of pip install for andes installation

conda install andes

or in the root directory of andes, run

conda install --file requirements.txt
conda install --file requirements-dev.txt
python3 -m pip install -e .

Test if andes has been successfully installed by

andes selftest

Install mpi4py from Intel with

conda install -c intel mpi4py

Install gym

pip install gym

Install a recent version of ANDES with

Install tensorflow (cpu version)

pip install andes==1.5.8

Install keras

pip install keras

Install stable-baselines or stable-baselines3 Refer to the stable-baselines website or stable-baselines3 for detailed instruction.

For stable-baselines

Option 1: First Homebrew will be needed. Then, install the dependencies of stable-baselines as

brew install cmake openmpi

Then, open the stable-baselines repo, cd to the directory and

pip install -e .

Option 2:

pip install stable-baselines

For stable-baselines3 （uses Pytorch 1.7）

pip install stable-baselines3

or in the root directory

pip install -e .

In the root directory of andes_gym , install andes_gym in the development mode with

pip install -e .

Try out examples in the folder examples .

Edit the number of time steps in train_freq_ddpg.py to a proper value. The larger the value, the longer it takes to simulate and train.

Run the example with

python train_freq_ddpg.py

Some deprecation warnings can be safely ignored.

When the training is completed, replay the trained model with

python play_freq_ddpg.py

You may need to install additional packages, such as matplotlib, to show the visualization. The matplotlib window might appear frozen, but is actually refreshed after each run.

While most of the computation time was spent in ANDES, tests show that Linux perform consistently better than Windows, especially with cvxoptklu installed.

Currently, it is difficult to install cvxoptklu on Windows. Therefore, a Linux box is recommended whenever possible.

Some algorithms can take advantage of multi-core processors but most cannot. Please check the algorithm documentation from stable-baselines to verify. A quick way to check if an algorithm is taking advantage of your multi-core processor is to check the utilization of CPU (in Windows Task Manager or in htop of Linux).

When working on the source code, please branch from master and work on your own branch.

You can either use GitHub for Desktop or learn the commands. The following are some commands for quick reference.

Branching can be done the collowing command

git checkout -b YOUR_BRANCH_NAME

where YOUR_BRANCH_NAME is the branch name of your choice.

To stage changes, use

git add PATH_TO_FILE

To commit changes, use

git commit

To push to a not-yet-exising branch, use

git push -u origin YOUR_BRANCH_NAME

After the first push, your local git will memorize the tracking branch. Next time, you can simply push with

git push

• TODO