This repository holds the work carried out for a project/assignment for the course of Seminar of Reinforcement Learning at Helsinki University. The final survey paper can be reviewed at the following file: Benchmarking and Reliability in RL.pdf, which includes the latest 2021 scientific works made on tesing and evaluating reliability of reinforcement learning model.

The absence of standardized methods and lack of adequate metrics for measur- ing Reliability and evaluating performances improvement of novel reinforcement learning (RL) is still a debated problem. We examine in this report past and re- cent approaches in order to give a summary of proposed solutions for addressing evaluation and comparison of different models. In particular, we describe metrics, methods, and statistical tests to follow for pertinent analysis, by describing their explanatory powers, usage, and related issues.