jnhwkim / cbp Goto Github PK

Hi,

I'm getting poor performance after loading a gmodule containing a cbp for classification.
It works well before I save(using torch.save) and load the module.

I'm thinking something is wrong with the parameters after loading, but I'm not sure what exactly are the module parameters, for my understanding we only sample h and s. Should I reset the module then after each iteration or keep them fixed? Am I missing something that might cause the loading problem?

Thanks!

Hi everyone, thank you for great code.
Can anyone tell me how to run the code with CPU only? I dont have GPU on my laptop and when I run the code, I got error "no CUDA-capable device is detected".
Thanks,
Kien.

hi, I come here because of your answer in the vqa-mcb.
I want to use cbp layer in torch for VQA, but the result is very poor. I am not sure it is whether my Programming errors or not.
Is there something need to pay attention to when we use cbp layer for VQA in torch?
Could you share your torch code with cbp layer for VQA ?
thanks!

Hi. This work CBP is very good at recognition and VQA on the papers.
The idea of feature learning by two-stream and merge them into one should be a general method.
However, I test CBP on local feature matching and the performance is very poor.
Have you test the method on other problem and the performance is not good?
Or is there some special criterion before I insert CBP into the network? (batch size? learning rate?)

To be more precise, I use the following network at the end of my network

local temp_m = nn.ConcatTable()

local A_net = nn.Sequential()
......
A_net:add(nn.Linear(4096,512))

local B_net = nn.Sequential()
......
B_net:add(nn.Linear(4096,512))

temp_m:add(A_net)
temp_m:add(B_net)
model:add(temp_m)
model:add(nn.CompactBilinearPooling(dim,true))
model:add(nn.SignedSquareRoot())
model:add(nn.Normalize(2))

Thank you for your reply.

Does current version only supports the bilinear pooling of two same-length vectors? Is it possible to support the pooling of (n, dimA) and (n, dimB) --> (n, compact_dimension) ?

I find this code can only handle the input whose dimension is 2.
in the paper of Berkeley, they use cbp layer to do attention, the input become 2048_14_14 or 2048_196, if the batchsize is 20, the input of cbp layer will become 20_2048_196 and the input dimension will become 3.

I find assert(false, '# of dimensions > 2') in CompactBilinearPooling.lua.
how to modify the code if the dimemsion of input is 3 or more ?

The gradient of Signed Square Root is 1 / (2 * sqrt(abs(x))). When x is close to zero, the gradient is explosive.

Refer to https://github.com/gy20073/compact_bilinear_pooling/blob/master/MatConvNet/layers/signed_sqrt/yang_sqrt_backward.m .