what is the meaning of each actions_set? about doom-net-pytorch HOT 9 CLOSED

This is a name of numpy file.
action_set = np.load('action_set_speed_shot_backward_right.npy')
Columns are actions as defined in available_buttons in .cfg file
Rows are actions. Each action is composed by combination of available_buttons. For example if
available_buttons = {
TURN_LEFT
TURN_RIGHT
ATTACK
}
then action = { 1, 0, 1 } means turn left and attack.

from doom-net-pytorch.

Thanks, in fact I want to run the deathmatch scenario by modifying the cig_aac.sh.
I have change the INSTANCE=cig to INSTANCE=basic, and CONFIG=$BASEDIR/environments/cig.cfg to CONFIG=$BASEDIR/environments/deathmatch.cfg.
But it can not get a decent agent. Should I change the actions_set or modify some other parameters?
Thanks!

from doom-net-pytorch.

Right, available_buttons buttons in deatchmatch.cfg do not match ones in cig.cfg, thus action_set is not applicable too. Try to set available_buttons the same as in cig.cfg.

from doom-net-pytorch.

Thanks a lot, I will try it. I noticed there is no actions_set in d3 scenario, so does it means the agent will take one action at a time when the actions_set is None?

from doom-net-pytorch.

Yes, one action at a time, here is the for action set in case of None

        if actions is None:
            self.actions = np.eye(len(self.game.get_available_buttons()), dtype=int).tolist()

from doom-net-pytorch.

Thanks, I have train the deathmatch scenario using d3_aac.sh, which the actions_set is None, and I get this after training for about 30000 iterations.
mean_return = 2.450000, grads_norm = 0.003205, weights_norm = 4.769231, batch_time = 0.818
the weights_norm and grads_norm has decreased a lot, but the reward did not increase at all. Should I do some adjustments or just train for a longer time.

from doom-net-pytorch.

AAC cannot handle D3 from scratch. I trained D3 with a prior initialization of weights from imitation policy - played few games and did supervised training of policy, see imitation.py.

If you don't want it, I suggest to try PPO, it may work. I just committed an update which has a new script d3_ppo.sh

from doom-net-pytorch.

One more thing - default rewards in D3 are not good, update doom_instance's step() and step_normalized() method like this:

    def step_normalized(self, action):
        state, reward, finished, dead = self.step(action)
        state = self.normalize(state)
               if state.variables is not None:
            diff = state.variables - self.variables
            diff[2] *= 100
            reward += diff.sum()
            self.variables = state.variables.copy()

        return state, reward, finished

    def step(self, action):
        ...
        finished = episode_finished or dead
        if finished:
            # self.episode_return = self.game.get_total_reward()
            self.episode_return = self.variables[2]
        ...

from doom-net-pytorch.

Actually, I just created a new instance for D3 config, just get the latest code, no need to change doom_instance.

from doom-net-pytorch.