richzhang / splitbrainauto Goto Github PK

In my reimplementation of the split brain experiment, I found the loss of the path L->ab convergences very slowly, but the loss of the path ab->L convergences quickly, is it normal? I used the regression loss in my reimplementation.

I am reimplementing the Split brain on my dataset, the my network is here:
the path of ab->L:(input:84,84,2)->(42,42,16)->(21,21,32)->(21,21,32)->(11,11,64), and the feature(11,11,64) is average pooled across channel, get a (11,11,1)map, than, the loss is the L2 loss between the pooled feature map (11,11,1) and the downsampled L channel of the original image (11,11,1), the downsample operation is (84,84,1)->(11,11,1),
I do not know whether my reimplementation is correct, please help me, thanks a lot!

As per the architecture, the CL/Reg loss is calculated after down sampling ground truth to 12 x 12 or upsampling 12 x 12 prediction to 180 x 180?

Also, I am not able to understand the reason for absorbing BatchNorm layer into conv layer after pretraining, is there any effect of this?

Thanks

Hi,
First of all, thank you for this repository, and please excuse me if the following questions are very basic.
I have started studying the splitbrain network model and reading the paper, but I have not yet understood where the encoding part ends and the decoding part starts in deploy_lab.prototxt. It seems to me that the output of the network gives a feature representation, and not the recreated image I expected to find. Could you please point me in the right direction?
Also, it seems to me that if you wanted to replicate the architecture described in your paper, you would need to use two networks. Is it correct?
Thank you in advance.

Thanks for your nice work! @richzhang
How can I use the official caffe to train your model with the lab color space input ? How to deal with the input in the provided prototxt(https://github.com/richzhang/splitbrainauto/blob/master/models/deploy_lab.prototxt)? Using the same RGB frames layer, such as ImageData Layer?

Hi, I don't quite understand the quantization procedure for classification loss. The L channel is quantized into 100 bins. The ab channels are quantized to the Q=313 values which are in-gamut..? Can you explain the procedure for identifying those values?

I quantize L_labels as np.digitize(image[:,:,0] , np.linspace(0,100,100))
How does it work for ab_labels..?

Secondly, is the regression loss is with respect to these quantized labels?

Apologies if these questions are too trivial.
Thanks! I found your work really fascinating!

Are fc1 and fc2 convolutional or fully connected layers. If they are in a fact convolutional layers please tell the number of filters.

Thanks

Hi!
I understand the split-brain encoder is formed by concatenating two sub-networks, and thereby the group of each convolution layer should be 2. However, some conv layers in the provided deploy.prototxt only have 1 group, such as conv1 and conv3.
Thanks.

In Table 4 for semantic segmentation on PASCAL VOC, mentions only for SplitBrain (cl,cl) loss. I am just wondering, did you tried with regression loss pre-trained network?