More information about the LongerExplorationPolicy about deer HOT 3 CLOSED

Hi dynamik,

Basically the idea is that if you have a pure random exploration, you will end up with all possible ordered sequences that have uniform probabilities. E.g., if a set of two possible actions {1,2} and two time steps, the sequences {11,12,21,22} have all the same probability 0.25 of being tried out.
For the LongerExplorationPolicy, the unordered sequences have uniform probabilities. So in the exemple, {11} has 0.33 probability, {22} has 0.33 and {12, 21} have together 0.33 (0.17 each). That can be useful in environment such as grid world where the order of the actions does not matter in most situations.

The length parameter should be chosen depending on your environment. Usually you'll have to try a few possibilities empirically and see what works.

Best,
Vincent

from deer.

Hi VinF,

thanks for your quick response!

How do you evaluate the Ornstein-Uhlenbeck-Process in comparison?

Best,
Roman

from deer.

Hi Roman,
Indeed, you could possibly find parallels with the nomenclature in the domain of stochastic processes depending on the setting considered.

from deer.