Pessimistic Agents are ask-for-help reinforcement learning agents that offer guarantees of:
- Eventually outperforming the mentor
- Eventually stopping querying the mentor
- Never causing unprecedented events to happen, with arbitrary probability
In this repository, we investigate their behaviour in the faithful setting, and explore approximations that allow them to be used in real-world RL problems.
Overview - see individual README.md files for more detail.
We introduce a tractable implementation of Pessimistic Agents. Approximate the Bayesian world and mentor models as a distribution over epistemic uncertainty of Q values. By using a pessimistic (low) quantily, we demonstrate the expected behaviour and safety results for a pessimistic agent.
Work | Status |
---|---|
Finite state Q Table proof of concept | |
Continuous deep Q learning implementation |
Implement and investigate a faithful representation of a Bayesian Pessimistic Agent.
Work | Status |
---|---|
Environment | |
Agent |
On hold, some progress made in implementing the environment and mentor models.
Apply pessimism approximation to neural network based, deep Q learning RL agents.
Work | Status |
---|---|
DQN proof of concept |
With anaconda
conda env create -f torch_env_cpu.yml