yijunmaverick / deepjointfilter Goto Github PK

I download the 30 middlebury images from
http://web.cecs.pdx.edu/~fliu/project/depth-enhance/
and use matlab code to downsample and upsample the images:

scale=8;
startpoint=scale;
low1= depth(startpoint:scale:end,startpoint:scale:end);
upsample1=double(imresize(low1,[height,width],'bicubic');

But the result is not good because the bicubic RMSE is 10.32 instead of 7.58 declared in the paper.
And i perform the same action on the Lu dataset, the RMSE is fine.
How should i perform the preprocess?

when I run examples/Test/cnn_test_upsampling.m, I get flowing error: References to nonexistent fields' dilate'(at line 303 in vl_simplenn.m). Does this mean I need a GPU to run the code? Thanks~

I have read the code carefully but can't find the skip connection part.
And i have several questions:

1> In the code , the input patch size is 33x33, but 32x32 in the paper?

2> In the paper with skip connection, is the output patch size the same with input, that is 33x33 ,different from the matlab implemention 21x21?

3>And it is strange: I have re-implement the no-skip connection version with pytorch and reach the RMSE of 5.9 (lower than 6.2 in the paper) , but when i add the skip connection with the training parameter unchanged (I set these parameters with the running data of the matlab code), the RMSE got worse. What should I do?

4>How can i further improve the RMSE?

Thanks.

Here is how i calculate the RMSE of bicubic method with Sun dataset:
0 make a mask with raw depth !=0
1 downsample depth_bfx with scale
2 upsample with matlab resize('bicubic')
3 res=depth_bfx - upsampled_depth
4 calcalute bicubic RMSE as : sum(res(mask))/sum(mask(:))

But the result is far worse from 3.45 of 8 scale in the paper ,so this leads to bad result in algorithm result.
For other datasets, the data is correct. What is the correct way to upsample the downsampled image and calculate RMSE in Sun?

Can you provide the evaluation metric that generate exactly result with the paper?
Here is my "bicubic" method and evaluation metric matlab code, but it has a little difference with the paper score: (example for Dataset Lu and scale 16)

scale=16;
count=0;
total2=0;
for i=1:6
depths_image=imread(sprintf('/home/xx/Depth_Enh/03_RGBD_Dataset/RGBD_0%d_output_depth.png',i));
rgb=imread(sprintf('/home/xx/Depth_Enh/03_RGBD_Dataset/RGBD_0%d_ouput_color.png',i));
depth=single(depths_image);
[hei,wid,channel]=size(depth);
max2=max(max(depth));
min2=min(min(depth));
depth = (depth-min2)/(max2-min2);
depthGT = uint8(depth255);
max1=255;
min1=0;
startpoint=scale;
low1=depth(startpoint:scale:end,startpoint:scale:end);
upsample1=double(imresize(low1,[hei,wid],'bicubic'));
padding=6
upsample2=upsample1(padding+1:hei-padding,padding+1:wid-padding)255;
depth2=depth(padding+1:hei-padding,padding+1:wid-padding)255;
res2=upsample2-depth2;
temp2=sum(sum(res2.^2));
num=(hei-2padding)(wid-2padding);
avg2=sqrt(temp2/num)
total2=total2+avg2;
end
total2/6