QLearning Can't Train On Episodes about mushroom-rl HOT 6 CLOSED

Given that there's an assertion, the behavior is actually the desired one.
Q learning is a step-based approach, and while the episodic version is possible to implement we deliberately chose not to due to simplicity.
The implementation of an episode base q-learning is trivial, and anyone can do it.

Please, avoid reporting bugs that are obviously invalid: if we put an assert there's a reason and it's not an error.

from mushroom-rl.

What is the point of making a library, restricting how agents can behave/learn and then telling people who ask for those limitations to be removed to go away?

I thought this was a cool library to try using for my research. What's the point in open-sourcing something and asking others to use it if you just tell them to go away?

from mushroom-rl.

We are more than open to requests and bug reports.

However, we cannot change the library following arbitrary requests from the user base, particularly if they break the design of the library. This has to do with software engineering issues: to maintain the library clean we have to impose design restrictions.

About the bug, we kindly ask to avoid reporting invalid bugs, as we are researchers with limited time and not a company. Please, in case the code throws an assertion error, the most likely reason is that we are restraining the behavior of the code on purpose.

from mushroom-rl.

Why does changing the implementation break the code? I would think there's a simple solution: remove the assert and loop over the dataset

from mushroom-rl.

the most likely reason is that we are restraining the behavior of the code on purpose.

Oh, most certainly. But from the outside, each assert is ambiguous whether it stems from the fact that (a) something cannot be done or (b) something is not yet implemented

from mushroom-rl.

In case something is not implemented, we will throw a not implemented error.

There is a reason why we put an assert, instead of a for loop: the most likely scenario is that a user will call wrongly the core function, and get an unexpected behavior from the algorithm.
In most scenarios, there's no reason to use a batch update. If the user wants to implement this behavior, it can create his own class.

The design rationale of mushroom is a clean implementation: we will not add any unnecessary options to the standard approaches. If the user wants a specific feature for an algorithm they can implement their version of the algorithm by copying the few lines of code and creating a new class.

from mushroom-rl.