312shan / pytorch-seq2seq-beam-search Goto Github PK

Hi 312shan, great work!

I was analyzing your code and was curious about parts of the forward for the Decoder and Attention module.

. In the forward function of the Attention Module, you are doing
h = hidden.repeat(timestep, 1, 1).transpose(0, 1) # [32, 512]=>[32, 27, 512]
since you have passed the last layer of hidden state from Decoder's forward function as
attn_weights = self.attention(last_hidden[-1], encoder_outputs) # [32, 512][27, 32, 512]=>[32, 1, 27]

However, why is only the last of the hidden used? I'm assuming it's because of meeting matrix dimensionality requirements for the Attention Module, but lets say you specified multi-layers for the Decoder Module such as ,

decoder = Decoder(embed_size, hidden_size, en_size, n_layers=2, dropout=0.0)

Would this be losing half of information from the hidden state, that the Attention Module could have possibly used since you are only taking the last layer of the hidden state into account everytime?

您好，我最近在看您代码Seq2Seq中的BeamSearch，请问beam_decode函数返回的decoded_batch是怎么运用的？
我只在测试的代码中看到有调用，但是没看到有后续的操作了

如上图，我本身target_sentence(包括SOS，EOS，pading)的长度设定的是8，但是我在测试的时候总会返回大量随机长度的tokens。请问如何解决

seq2seq.decode() 方法没return beam 或 greedy decode 的结果