(This current quickstart assumes a Unix-based OS, if you're on Windows, please adapt the steps to the appropriate Windows analog.)

Clone the repo to your local machine.

If you're authorizing the git cli with an app password, you can use HTTPS:

git clone https://github.com/ankile/dtak-mhealth-rl.git

Otherwise, SSH is probably advisable:

git clone [email protected]:ankile/dtak-mhealth-rl.git

Make a virtual environment

Make sure you have Python 3 installed (preferably 3.10 or above). If you do not, may I suggest using Homebrew (brew.sh) to install it?

To create the virtual environment, run (I prefer to use python3.11 to ensure what version we're using):

python3 -m venv venv

Activate the environment

source venv/bin/activate && pip install --upgrade pip

(This command also upgrades pip because it always complains about being outdated, so why wait?)

Install the required packages

pip install -r requirements.txt

Run experiments

If you want to run, let's say, the perturbation plot for the Cliff world, you can first edit the relevant fields in the src/param_sweeps/cliff_world.py:

  run_parallel = True

  # Set the number of subplots per row
  cols = 4  # 5, 7, 9

  # Define the default parameters
  default_params = {
      "height": 5,
      "width": 9,
      "reward_mag": 1e2,
      "neg_mag": -1e8,
      "latent_reward": 0,
  }

  # Define the search space
  search_parameters = {
      "width": np.linspace(4, 10, cols).round().astype(int),
      "height": np.linspace(4, 10, cols).round().astype(int),
      "reward_mag": np.linspace(100, 500, cols),
  }

  # Set the number of scales and gammas to use
  granularity = 20  # 5, 10, 20

  # Set up parameters to search over
  probs = np.linspace(0.4, 0.99, granularity)
  gammas = np.linspace(0.4, 0.99, granularity)

When you're happy with all the settings, run the code in the if __name__ == "__main__": block:

python -m src.param_sweeps.cliff_world

(Mark the lack of .py at the end.)

Next steps

Happy hacking!