In this repository we provide training pipelines for gymnax agents in JAX. Furthermore, you can use the utilities to speed benchmark step transitions and to visualize trained agent checkpoints. Install all required dependencies via:

pip install -r requirements.txt

We provide training routines and the corresponding checkpoints for both Evolution Strategies (mostly OpenES using evosax and PPO (using an adaptation of @bmazoure's implementation). You can train the agents as follows:

python train.py -config agents/<env_name>/ppo.yaml
python train.py -config agents/<env_name>/es.yaml

This will store checkpoints and training logs as pkl in agents/<env_name>/ppo.pkl. Collect all training runs sequentially via:

bash exec.sh train

You can also generate GIF visualizations of the trained agent's behaviour as follows:

python visualize.py -env <env_name> -train <{es/ppo}>

Collect all visualizations sequentially via:

bash exec.sh visualize

Note that we do not support visualizations for most behavior suite environments, since they do not lend themselves to visual display (e.g. markov reward process, etc.).

Finally, we provide simple tools for benchmarking the speed of step transitions on different hardware (CPU, GPU, TPU). For a specific environment the speed estimates can be obtained as follows:

python speed.py -env <env_name> --use_gpu --use_network --num_envs 10

Collect all speed estimates sequentially via:

bash exec.sh speed

For each environment we estimate the seconds required to execute 1 Mio steps on various hardware and for both random (R)/neural network (N) policies. We report the mean over 10 independent runs:

TPU v3-8 comparisons will follow once I (or you?) find the time and resources 🤗