A modern & clean implementation of the PILCO Algorithm in TensorFlow.

Unlike PILCO's original implementation which was written as a self-contained package of MATLAB, this repository aims to provide a clean implementation by heavy use of modern machine learning libraries.

In particular, we use TensorFlow to avoid the need for hardcoded gradients and scale to GPU architectures. Moreover, we use GPflow for Gaussian Process Regression.

The core functionality is tested against the original MATLAB implementation.

1. Install venv

virtualenv -p python3 venv
source venv/bin/activate

2. Install requirements

pip install -r requirements.txt
python setup.py develop

3. You might also need to install openai gym

pip install gym

4. You might also need to install mujoco click here
5. In the case that mujoco and mujoco-py fail to build on macos, change the MacOS SDK to 10.14.sdk

Before using, or installing, PILCO, you need to have Tensorflow 1.13.1 installed (either the gpu or the cpu version). It is recommended to install everything in a fresh conda environment with python>=3.7. Given Tensorflow, PILCO can be installed as follows

git clone https://github.com/nrontsis/PILCO && cd PILCO
python setup.py develop

The examples included in this repo use OpenAI gym 0.15.3 and mujoco-py 2.0.2.7. Once these dependencies are installed, you can run one of the examples as follows

python examples/inverted_pendulum.py

While running an example, Tensorflow might print a lot of warnings, some of which are deprecated. If necessary, you can suppress them by running

tf.logging.set_verbosity(tf.logging.ERROR)

right after including TensorFlow in Python.

The following people have been involved in the development of this package:

• Nikitas Rontsis
• Kyriakos Polymenakos

See the following publications for a description of the algorithm: 1, 2, 3