Hi @ResearchingDexter , there might be something wrong with the

a_b=list(a_b.chunk(self.M,dim=1))
And it always give me this error unless I turn of cudnn which makes it very slow:
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_SUPPORTED. This error may appear if you passed in a non-contiguous input.

Any suggestion?

the dimension d is decided by the code line 6: d=max(in_channels//r,L)
But in the original paper, d = max(C/r;L);
Shouldn't "C" equals to "out_channels" instead of "in_channels"?
Small question, Thanks.

Hi. In the part of selection, the original paper wrotes that

In the case of two branches, the matrix B is redundant because ac + bc = 1.

but in your code:
a_b=self.fc2(z) a_b=a_b.reshape(batch_size,self.M,self.out_channels,-1) a_b=self.softmax(a_b) #the part of selection a_b=list(a_b.chunk(self.M,dim=1))#split to a and b
I did not see any constraints to a and b to ensure that a + b = 1. Does this impact the performance of the network?
Thanks.

hello @ResearchingDexter
for fully connected (fc) layer implement, why not use nn.Linear() to do,
i think that nn.Linear() -->bn --> relu, and why did you use bias=false in your conv2d?
Thanks, look forward to your early reply.

hello @bofang Liu:
for fully connected (fc) layer implement, why not use nn.Linear() to do,
i think that nn.Linear() -->bn --> relu, and why did you use bias=false in your conv2d?
Thanks, look forward to your early recovery.