Multiple policy heads in RNaD about open_spiel HOT 4 CLOSED

I'm not sure, I will check with @perolat and come back to this.

from open_spiel.

I believe we have already answered this. Can you check if #1053 answers your question?

from open_spiel.

@lanctot thanks for the reply and pointing to the past discussions.

It partialy helps us. But we are in the same situation presented in the last comment in #1053; we have multiple prediction heads in order to deconstruct the wide action space: there's one head for the specific action type, one which selects the unit that should do the action (some actions require a unit, others don't), one which selects a target unit and location, etc.; there are a total of 5 heads where the output from one head is used as input for another head and so on. This is what leads to confusion when applying V-Trace to this type of deconstructed policy.

To relate to RNaD in Stratego, it's clear you use one head in each specific phase of the game and only one value head. But you also have the case where you select a piece based on one policy head and then use its output to determine the piece destination from another policy head. How is this reflected in the game trajectory? Are there two consecutive steps each belonging to a policy head but for the same player?

from open_spiel.

Hi,

You got it right, there are just two consecutive steps for the same player (one to select the piece and one to move it) during the gameplay phase. The head is then selected accordingly.

Hope this helps,
Julien

from open_spiel.