Comments (34)
Could you please raise a question more specifically like which part of the training pipeline you do not fully understand?
from updet.
The 5v5 scenario is transferred to the 3v3 scenario. The input and output of the model have changed. Can't the model be retrained? How can we achieve the transfer of different tasks? Or what parameters need to be changed during migration? Which parameters are fixed?
from updet.
Which part of the code does the left half of Figure 3 correspond to?
from updet.
7m-5m-3m transfer learning,The observation spatial scale of each task changes, and the action spatial scale changes. The input and output of Updet model have changed. Doesn't the model need to be retrained?
from updet.
How can the model trained in the 5v5 scenario be used in the 3v3 scenario? Can I use it directly by changing the input and output of the model?
from updet.
You can train a 5m model and directly apply the saved model to 8m without any modifications. Note that only agent model can be transferred not the centralized function except VDN.
from updet.
For how it works, please refer to our paper. The math formulations of how transformer works in multiagent system is described clearly. For implement details, you could refer to updet_agent.py.
from updet.
You can train a 5m model and directly apply the saved model to 8m without any modifications.
The input and output of the model have changed, how to use it directly? Don't need to retrain the model
from updet.
updet_agent.py,input_shape,不同场景,这里的input_shape也是需要改变的
In different scenarios, the input_shape here also needs to be changed
class UPDeT(nn.Module):
def __init__(self, input_shape, args):
super(UPDeT, self).__init__()
self.args = args
self.transformer = Transformer(args.token_dim, # 'token_dim': 5,
args.emb, # 'emb': 32,
args.heads, # 'heads': 3,
args.depth, # 'depth': 2,
args.emb) # 'emb': 32,
self.q_basic = nn.Linear(args.emb, 6)
def init_hidden(self):
return torch.zeros(1, self.args.emb).cuda()
def forward(self, inputs, hidden_state, task_enemy_num, task_ally_num):
outputs, _ = self.transformer.forward(inputs, hidden_state, None)
from updet.
You can train a 5m model and directly apply the saved model to 8m without any modifications.
Can the observation space size and action space size of the two scenes be different, can they be used directly?
from updet.
do you understand me
from updet.
Transformer arch can accept various token numbers. That’s why. Please run the code as what I said before and you will understand.
from updet.
You can train a 5m model and directly apply the saved model to 8mHow to use it directly? Is obs size and action size different? without any modifications.
inputs need change?
from updet.
In other words, the transformer can be migrated. In different task scenarios, the input and output have to be changed.
Does your article achieve this effect?
from updet.
Just this one, do migrations on different tasks
self.transformer = Transformer(args.token_dim, # 'token_dim': 5,
args.emb, # 'emb': 32,
args.heads, # 'heads': 3,
args.depth, # 'depth': 2,
args.emb) # 'emb': 32,
from updet.
yes or no
from updet.
Changes in inputs will not affect the migration of the transformer,transformer Can be reused
from updet.
You can train a 5m model and directly apply the saved model to 8m without any modifications.
The transformer can be reused, but the input and output still need to be changed.
from updet.
Please refer to build_input_transformer function in basic_controller.py. Inputs are divided into tokens according to agent number. As I said , please first run the code and then ask questions. Things are clear in code.
from updet.
I read the code, but my level is low and I didn’t understand it
inputs.append(th.zeros_like(batch["actions_onehot"][:, t]))
else:
inputs.append(batch["actions_onehot"][:, t-1])
if self.args.obs_agent_id:
inputs.append(th.eye(self.n_agents, device=batch.device).unsqueeze(0).expand(bs, -1, -1))
inputs = th.cat([x.reshape(bs*self.n_agents, -1) for x in inputs], dim=1)
return inputs
def _build_inputs_transformer(self, batch, t):
# currently we only support battles with marines (e.g. 3m 8m 5m_vs_6m)
# you can implement your own with any other agent type.
inputs = []
# print('batch["obs"]:', batch["obs"].shape)
# batch["obs"]: torch.Size([1, 71, 5, 55]) # 1 batch_size = 1, step = 71
raw_obs = batch["obs"][:, t]
# print('raw_obs:', raw_obs.shape)
# raw_obs: torch.Size([1, 5, 55]) 一步
# print('t:', t)
arranged_obs = th.cat((raw_obs[:, :, -1:], raw_obs[:, :, :-1]), 2)
# print('(raw_obs[:, :, -1:]:', raw_obs[:, :, -1:].shape)
# print('raw_obs[:, :, :-1]:', raw_obs[:, :, :-1].shape)
# (raw_obs[:, :, -1:]: torch.Size([1, 5, 1])
# raw_obs[:, :, :-1]: torch.Size([1, 5, 54])
# print('------------arranged_obs:', arranged_obs.shape)
# arranged_obs: torch.Size([1, 5, 55])
# view在pytorch中view函数的作用为重构张量的维度,相当于numpy中resize()的功能
from updet.
reshaped_obs = arranged_obs.view(-1,
1+(self.args.enemy_num-1)+self.args.ally_num,
self.args.token_dim)
不同任务场景,enemy_num 和 ally_num 也是不一样的啊
5m->8m 这个不需要改变吗
In different mission scenarios, enemy_num and ally_num are also different.
5m->8m does this need to be changed
from updet.
sorry I can’t teach you how to read the code. I recommend you to use an IDE like pycharm and debug the code step by step. You could access all the variables in the debugging process and see how the updet works. For ally and enemy number varying from 5m to 8m, that’s exactly how updet transfer to different tasks as updet accepts different token number we have built from raw input, which is a vector in GRU based model.
from updet.
最后一个问题,5m到8m做迁移学习,那一部分是复用的?transformer模块吗
The last question is, 5m to 8m do transfer learning, which part is reused? transformer module?
from updet.
最后一个问题,5m到8m做迁移学习,那一部分是复用的?transformer模块吗
The last question is, 5m to 8m do transfer learning, which part is reused? transformer module?
from updet.
最后一个问题,5m到8m做迁移学习,那一部分是复用的?transformer模块吗
The last question is, 5m to 8m do transfer learning, which part is reused? transformer module?
from updet.
最后一个问题,5m到8m做迁移学习,那一部分是复用的?transformer模块吗
The last question is, 5m to 8m do transfer learning, which part is reused? transformer module?
from updet.
最后一个问题,5m到8m做迁移学习,那一部分是复用的?transformer模块吗
The last question is, 5m to 8m do transfer learning, which part is reused? transformer module?
from updet.
Yes. As it is the fixed part of updet, it can be reused. But if you are interested in applying transformer variants to updet, you can modify this part.
from updet.
我想在coma上 使用这个updet可以吗
I want to use this updet on coma, is it okay?
from updet.
我想在coma上 使用这个updet可以吗
I want to use this updet on coma, is it okay?
from updet.
我想在coma上 使用这个updet可以吗
I want to use this updet on coma, is it okay?
from updet.
我想在coma上 使用这个updet可以吗
I want to use this updet on coma, is it okay?
from updet.
我想在coma上 使用这个updet可以吗
I want to use this updet on coma, is it okay?
from updet.
我想在coma上 使用这个updet可以吗
I want to use this updet on coma, is it okay?
from updet.
Related Issues (17)
- Release Code HOT 1
- TypeError: expected str, bytes or os.PathLike object, not NoneType HOT 2
- question about 7m-5m-3m transfer learning HOT 3
- Why does updet realize generalization? not understand,
- Why does updet realize generalization? not understand,
- Why does updet realize generalization? not understand,
- 我想在coma上 使用这个updet可以吗 I want to use this updet on coma, is it okay? HOT 10
- 代码调试问题 HOT 7
- Shouldn't pygame's version be 2.0.0? HOT 1
- In which part do you implement policy decoupling HOT 3
- In which part observation-entity and action-groups match HOT 1
- Unsatisfactory experimental results HOT 2
- How to reproduce the transfer learning HOT 4
- Can you send this map? HOT 1
- action group
- the drawing code
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from updet.