how to speed up to model pruned? about pytorch-pruning HOT 7 OPEN

The ResNet model pruned impletement faster little about 10%, code in
https://github.com/eeric/channel_prune

the reason that non-tensor layers (e.g., batch normalization
and pooling layers) took up more than 40% of the inference
time on GPU.

from pytorch-pruning.

How to get this "the reason that non-tensor layers (e.g., batch normalization
and pooling layers) took up more than 40% of the inference
time on GPU." Are there some papers or other things?
@eeric

from pytorch-pruning.

test myself

from pytorch-pruning.

Hey Guys, I pruned the SqueezeNet model and when I test for the inference time, it is same on the GPU. I can see the difference on� CPU(after 67% filter pruning, inference time is almost half) but, on GPU it is same. Any Idea why this is happening?
After Pruning:

• Model Size reduced
• I can see the difference in FLOP
• Pruned model is faster on CPU but not on GPU. :(

Thank you!!

from pytorch-pruning.

@eeric Actually, after pruning, the number of input channel and output channel is no longer integer multiple of 32. When cuda compute the convolution, it actually transforms it into matrix mulitiplication. The warp of cuda is 32.

from pytorch-pruning.

@buaaJeremyduan, thanks!

from pytorch-pruning.

@Kuldeep-Attri did you find out the reason?
My observation in that regard is same. Let me know if you have any leads?

from pytorch-pruning.