why don't consider relu layer in CR? about channel-pruning HOT 15 CLOSED

It's just a matter of notation. Using conv makes it easier to prune conv filters

from channel-pruning.

I don't undstand why not use relu in CR stage , but use relu in VH stage ?

But I want to know whether I should consider relu in CR in single CR / single channel pruning?
thank you very much for your answer!

from channel-pruning.

I'm not clear what you mean. How to pruning a single channel layer?

from channel-pruning.

I mean only CR , no VH and ITQ

from channel-pruning.

I still don't understand

• consider relu in CR
• why not use relu in CR stage , but use relu in VH stage

from channel-pruning.

Debug info:
In CR stage:
Extracting X conv3_1 From Y conv3_2 stride 1
In VH stage:
Extracting X relu3_1 From Y conv3_2 stride 1

I don't understand why don't consider relu3_1 in CR stage ?
my QQ number is : 317378808
Can we communicate with QQ, thank you very much

from channel-pruning.

Please read 3C approach in 4.1.2, and factorization papers [22,53]

from channel-pruning.

'''CR''' stage
if conv in pooldic:
X_name = self.bottom_names[convnext][0]
else:
X_name = conv
X_name 不应该全部都采用self.bottom_names[convnext][0]吗？
如果仅仅使用CR，没有 VH 和 ITH；
X_name 应该怎么输入？

from channel-pruning.

用 self.bottom_names[convnext][0] 是因为conv之间隔了一个pooling

from channel-pruning.

thank you very much!

from channel-pruning.

I have sended a message to your edu mailbox
I'm glad you could answer

from channel-pruning.

@yihui-he, I'm sorry for keep asking, but I don't know Chinese and google's translation is bad.
My question is:

If we only execute the Channel Pruning stage, do we need to extract all ReLU and all Pool Layers from all the CONV layers?

I'm confused because like @zlheos, I also observed that different pruning stages extract different layers, for example:

First Iteration
----------------------------------------------
Extracting X relu1_1 From Y conv1_2 stride 1
spatial_decomposition 90.12070608139038
----------------------------------------------
Extracting conv1_2 (50000, 64) -- samples
channel_decomposition 66.06586813926697
------------------------------------------------
Extracting X pool1 From Y conv2_1 stride 1
channel_pruning 46.22542428970337
------------------------------------------------
Second Iteration
------------------------------------------------
Extracting X pool1 From Y conv2_1 stride 1
spatial_decomposition 81.37090802192688
------------------------------------------------
Extracting conv2_1 (50000, 128) -- samples
channel_decomposition 96.16854739189148
------------------------------------------------
Extracting X conv2_1 From Y conv2_2 stride 1
channel_pruning 63.9356963634491

But the description of extract_XY() says it only supports conv-relu-conv operation:

https://github.com/yihui-he/channel-pruning/blob/3a4614de8c11aeb2ab981ddc619927c0e60a6577/lib/net.py#L532-L539

from channel-pruning.

yes you need to extract pooling

from channel-pruning.

@yihui-he but we don't care about ReLU because we already split it in step0, correct?

from channel-pruning.

from channel-pruning.