Representing activity-based modeling problems as reinforcement learning problems using the gymnasium api.

• biggym.envs.SchedulerEnv: Simple "scheduling" challenge for a single agent with minimal interaction with world.

  • agent has three possible states: at home, at work, traveling
  • agent has two possible actions: travel to work, travel to home
  • reward is based on MATSim utility score

• biggym.envs.SchedulerModeEnv: Extends above to include an additional activity "shop" and travel mode choice between car, bus and walk.

  • agent has 6 possible states: at home, work, shop and traveling by car, bus or walking
  • agent has 9 possible actions, travel to home, work or shop, each by either car, bus or walk
  • reward is based on MATSim utility score

• how to deal with noops
• how to deal with multi-dim actions, eg participation and mode

• allow additional activity choices (participation, location, time, mode, toll routing, charging, routing)
• separate choice dimensions better (eg participation and mode as different actions spaces)
• add stochastics to travel times
• multi-day
• create continuous choices (ie "go to work for N hours")
• inverse RL
• electric vehicle charging

To install we recommend using the mamba package manager.

git clone [email protected]:arup-group/biggym.git
cd biggym
mamba create -n biggym --file requirements/base.txt --file requirements/dev.txt
mamba activate biggym
pip install --no-deps -e .

To run the example notebooks you will need to add a ipython kernel into the mamba environemnt: ipython kernel install --user --name=biggym.