question about resnet-1k-layers HOT 3 CLOSED

This is realized in:
https://github.com/KaimingHe/resnet-1k-layers/blob/master/resnet-pre-act.lua#L65-L66
to which the first residual unit belongs.

Alternatively, you may refer to the 'first' option in fb-torch-resnet's pre-resnet implementation:
https://github.com/facebook/fb.resnet.torch/blob/master/models/preresnet.lua#L147-L150
which makes it explicit.

from resnet-1k-layers.

Oh, I know that. Thanks very much!

from resnet-1k-layers.

Hi,

I’m not sure this is the best place to post this, since this thread is more than 3 years old, but I am struggling to understand the exact implementation used in the paper. Clarification would be greatly appreciated!

As far as I can tell, the two implementations you have linked above are not identical. My understanding of what happens in the 2 implementations is as follows:

• The first implementation (https://github.com/KaimingHe/resnet-1k-layers/blob/master/resnet-pre-act.lua#L65-L66) results in activations of the shortcut path for the first residual unit of each (!) of the 3 stages in the entire network (each time a residual unit is used for increasing dimensions, which happens 3 times in total).
• The second implementation applies the activation after conv1 and before any of the residual stages (https://github.com/facebook/fb.resnet.torch/blob/master/models/preresnet.lua#L147-L150). This would correspond to my interpretation of what I read in the paper.

“For the first Residual Unit (that follows a stand-alone convolutional layer, conv1), we adopt the first activation right after conv1 and before splitting into two paths;”

Did I understand this correctly? I am not familiar with the syntax used in this version of torch, so that might have caused me to miss something.

Secondly, in the facebook implementation, this initial activation is implemented only for ImageNet, not for CIFAR (https://github.com/facebook/fb.resnet.torch/blob/a96f50e9041b3531f90ebe768f6ddfc49c26f56d/models/preresnet.lua#L166). The activation after the last residual block is, however, used in both cases (https://github.com/facebook/fb.resnet.torch/blob/a96f50e9041b3531f90ebe768f6ddfc49c26f56d/models/preresnet.lua#L172).

Thanks in advance,
Nicholas

from resnet-1k-layers.