Code of Dealing with Multiple Experts and Non-Stationarity in Inverse Reinforcement Learning : An Application to Real-Life Problems

This repository contains the implementation of the SIGMA-GIRL algorithm and experiments in real-world data.

• SIGMA-GIRL
• MI-SIGMA-GIRL
• NS-SIGMA-GIRL

To reproduce our results run the following scripts for each use case:

• The notebook 'notebooks/Highway_use_case' contains the visualization of all the results in the paper together with references to the scripts to run before executing the notebook cells
• The notebook 'notebooks/Highway_training_curves' contains the visualization of the training curves of the policies trained with the recovered cluster reward functions. It takes in input the training logs, pregenerated and uploaded in the repo (logs/highway/trained). We cannot provide the simulator hence the source code for training, but details can be found in https://www.sciencedirect.com/science/article/abs/pii/S0921889020304085

• To process the raw data into state, action reward tuples execute the 'notebooks/Data Preprocessing Twitter' notebook (reads the raw data found in datasets/twitter)
• Most of the results of the twitter use case can be reproduced in the repository where Sigma-GIRL was first introduced (https://github.com/gioramponi/sigma-girl-MIIRL)
• Two extra experiments performed in this paper can be visualized in the notebook 'notebooks/Twitter_use_case' together with references to the scripts to run before executing the notebook cells

- The raw data of the Como Use Case cannot be made available at the moment as they are propietary
+ We provide though the implementation of NS-SIGMA-GIRL as well as the output of running it on the Como Lake data
+ to reproduce the Figures shown in the paper by running the Notebook

• The notebook 'notebooks/Como Use Case' contains the presentation of the results generated by running 'run_change_detection_como.py'