Why does updet realize generalization? not understand, about updet HOT 34 CLOSED

Could you please raise a question more specifically like which part of the training pipeline you do not fully understand?

from updet.

The 5v5 scenario is transferred to the 3v3 scenario. The input and output of the model have changed. Can't the model be retrained? How can we achieve the transfer of different tasks? Or what parameters need to be changed during migration? Which parameters are fixed?

from updet.

Which part of the code does the left half of Figure 3 correspond to?

from updet.

7m-5m-3m transfer learning，The observation spatial scale of each task changes, and the action spatial scale changes. The input and output of Updet model have changed. Doesn't the model need to be retrained?

from updet.

How can the model trained in the 5v5 scenario be used in the 3v3 scenario? Can I use it directly by changing the input and output of the model?

from updet.

You can train a 5m model and directly apply the saved model to 8m without any modifications. Note that only agent model can be transferred not the centralized function except VDN.

from updet.

For how it works, please refer to our paper. The math formulations of how transformer works in multiagent system is described clearly. For implement details, you could refer to updet_agent.py.

from updet.

You can train a 5m model and directly apply the saved model to 8m without any modifications.

The input and output of the model have changed, how to use it directly? Don't need to retrain the model

from updet.

updet_agent.py，input_shape，不同场景，这里的input_shape也是需要改变的
In different scenarios, the input_shape here also needs to be changed

class UPDeT(nn.Module):

def __init__(self, input_shape, args):
    super(UPDeT, self).__init__()
    self.args = args

    self.transformer = Transformer(args.token_dim, # 'token_dim': 5,
                                   args.emb, # 'emb': 32,
                                   args.heads, # 'heads': 3,
                                   args.depth, # 'depth': 2,
                                   args.emb) # 'emb': 32,

    self.q_basic = nn.Linear(args.emb, 6)

def init_hidden(self):
    return torch.zeros(1, self.args.emb).cuda()

def forward(self, inputs, hidden_state, task_enemy_num, task_ally_num):
   outputs, _ = self.transformer.forward(inputs, hidden_state, None)

from updet.

You can train a 5m model and directly apply the saved model to 8m without any modifications.

Can the observation space size and action space size of the two scenes be different, can they be used directly?

from updet.

do you understand me

from updet.

Transformer arch can accept various token numbers. That’s why. Please run the code as what I said before and you will understand.

from updet.

You can train a 5m model and directly apply the saved model to 8mHow to use it directly? Is obs size and action size different? without any modifications.

inputs need change?

from updet.

In other words, the transformer can be migrated. In different task scenarios, the input and output have to be changed.

Does your article achieve this effect?

from updet.

Just this one, do migrations on different tasks
self.transformer = Transformer(args.token_dim, # 'token_dim': 5,
args.emb, # 'emb': 32,
args.heads, # 'heads': 3,
args.depth, # 'depth': 2,
args.emb) # 'emb': 32,

from updet.

yes or no

from updet.

Changes in inputs will not affect the migration of the transformer，transformer Can be reused

from updet.

You can train a 5m model and directly apply the saved model to 8m without any modifications.

The transformer can be reused, but the input and output still need to be changed.

from updet.

Please refer to build_input_transformer function in basic_controller.py. Inputs are divided into tokens according to agent number. As I said , please first run the code and then ask questions. Things are clear in code.

from updet.

I read the code, but my level is low and I didn’t understand it
inputs.append(th.zeros_like(batch["actions_onehot"][:, t]))
else:
inputs.append(batch["actions_onehot"][:, t-1])
if self.args.obs_agent_id:
inputs.append(th.eye(self.n_agents, device=batch.device).unsqueeze(0).expand(bs, -1, -1))

    inputs = th.cat([x.reshape(bs*self.n_agents, -1) for x in inputs], dim=1)
    return inputs

def _build_inputs_transformer(self, batch, t):

    # currently we only support battles with marines (e.g. 3m 8m 5m_vs_6m)
    # you can implement your own with any other agent type.

    inputs = []

    # print('batch["obs"]:', batch["obs"].shape)
    # batch["obs"]: torch.Size([1, 71, 5, 55]) # 1 batch_size = 1, step =  71

    raw_obs = batch["obs"][:, t]

    # print('raw_obs:', raw_obs.shape)
    # raw_obs: torch.Size([1, 5, 55]) 一步
    # print('t:', t)

    arranged_obs = th.cat((raw_obs[:, :, -1:], raw_obs[:, :, :-1]), 2)

    # print('(raw_obs[:, :, -1:]:', raw_obs[:, :, -1:].shape)
    # print('raw_obs[:, :, :-1]:', raw_obs[:, :, :-1].shape)
    # (raw_obs[:, :, -1:]: torch.Size([1, 5, 1])
    # raw_obs[:, :, :-1]: torch.Size([1, 5, 54])

    # print('------------arranged_obs:', arranged_obs.shape)
    # arranged_obs: torch.Size([1, 5, 55])

    # view在pytorch中view函数的作用为重构张量的维度，相当于numpy中resize（）的功能

from updet.

reshaped_obs = arranged_obs.view(-1,
1+(self.args.enemy_num-1)+self.args.ally_num,
self.args.token_dim)

不同任务场景，enemy_num 和 ally_num 也是不一样的啊
5m->8m 这个不需要改变吗
In different mission scenarios, enemy_num and ally_num are also different.
5m->8m does this need to be changed

from updet.

sorry I can’t teach you how to read the code. I recommend you to use an IDE like pycharm and debug the code step by step. You could access all the variables in the debugging process and see how the updet works. For ally and enemy number varying from 5m to 8m, that’s exactly how updet transfer to different tasks as updet accepts different token number we have built from raw input, which is a vector in GRU based model.

from updet.

最后一个问题，5m到8m做迁移学习，那一部分是复用的？transformer模块吗
The last question is, 5m to 8m do transfer learning, which part is reused? transformer module?

from updet.

最后一个问题，5m到8m做迁移学习，那一部分是复用的？transformer模块吗
The last question is, 5m to 8m do transfer learning, which part is reused? transformer module?

from updet.

最后一个问题，5m到8m做迁移学习，那一部分是复用的？transformer模块吗
The last question is, 5m to 8m do transfer learning, which part is reused? transformer module?

from updet.

最后一个问题，5m到8m做迁移学习，那一部分是复用的？transformer模块吗
The last question is, 5m to 8m do transfer learning, which part is reused? transformer module?

from updet.

最后一个问题，5m到8m做迁移学习，那一部分是复用的？transformer模块吗
The last question is, 5m to 8m do transfer learning, which part is reused? transformer module?

from updet.

Yes. As it is the fixed part of updet, it can be reused. But if you are interested in applying transformer variants to updet, you can modify this part.

from updet.

我想在coma上 使用这个updet可以吗
I want to use this updet on coma, is it okay?

from updet.

我想在coma上 使用这个updet可以吗
I want to use this updet on coma, is it okay?

from updet.

我想在coma上 使用这个updet可以吗
I want to use this updet on coma, is it okay?

from updet.

我想在coma上 使用这个updet可以吗
I want to use this updet on coma, is it okay?

from updet.

我想在coma上 使用这个updet可以吗
I want to use this updet on coma, is it okay?

from updet.

我想在coma上 使用这个updet可以吗
I want to use this updet on coma, is it okay?

from updet.