hezhangsprinter / did-mdn Goto Github PK

I am getting a cropped output derained image, as the program in test.py seems to be providing a cropped version of the input image to the network, even if the input image size is (512,512,3).
Actual input image(size 512):

The input to the network seems to be something like this:

I have added the following code snippet in test.py for figuring out what the input is:
val_target_cpu, val_input_cpu = val_target_cpu.float().cuda(), val_input_cpu.float().cuda() val_batch_output = torch.FloatTensor(val_input.size()).fill_(0) #added by me print(val_input_cpu[0,:,:,:].size()) x=val_input_cpu[0,:,:,:] print(x.size()) x=x.data.cpu().permute(1,2,0) print(x.size()) x=x.mul(255).clamp(0,255).byte().numpy()[:,:,:] print(x.shape) filename='./result_all/new_model_data/testing_our_our/'+str(i)+'_input.jpg' i=Image.fromarray(x) i.save(filename) #ends here val_input.resize_as_(val_input_cpu).copy_(val_input_cpu) val_target=Variable(val_target_cpu, volatile=True)
The output is:

from test.py, it seems that there three sub-networks (Dense_rain_residual, vgg19ca, Dense_rain) are actually used, which are completely different architecture.
It was never mentioned in the paper.

i found i cant change batchSize,when i change valBatchSize the GPU memory dont increase. @hezhangsprinter how can i do?

When i run test.py ,the error came out:

Traceback (most recent call last):
File "test.py", line 156, in
net_label.load_state_dict(torch.load('./classification/netG_epoch_9.pth'))
File "E:\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 522, in load_state_dict.format(name))
KeyError: 'unexpected key "conv_refin.weight" in state_dict'.

How can I solve the problem?

Hi, @hezhangsprinter . When I use the code to test my test dataset, it seems to resize the image to 512512. I tried to modify the orignalSize and imageSize of inputs, while there occurred some errors。I want to figure out whether the code can only tackle with image size of 512512, and the other size of image must resize/crop to 512*512. Because I want to compare your work with ours, I must test the other images. I am looking forward to your answer, thanks a lot.

how to handle this issue?

Hi, Zhang. I want to download your dataset. However, I found it was not accessible. Could your please fix it? Thanks.

Thanks for sharing for your code. I have one problem：
the readme show this method can process light、medium、heavy rain, but the training example is only Rain_Medium，if i want to train light、medium、heavy rain，Do I
python derain_train_2018.py --dataroot ./facades/DID-MDN-training/Rain_Light/train2018new --valDataroot ./facades/github --exp ./check --netG ./pre_trained/netG_epoch_9.pth.
python derain_train_2018.py --dataroot ./facades/DID-MDN-training/Rain_Heavy/train2018new --valDataroot ./facades/github --exp ./check --netG ./pre_trained/netG_epoch_9.pth.
python derain_train_2018.py --dataroot ./facades/DID-MDN-training/Rain_Medium/train2018new --valDataroot ./facades/github --exp ./check --netG ./pre_trained/netG_epoch_9.pth.
？
waiting for your answers.thanks very much！

Code snippet from train_rain_class.py:

netG = net1.vgg19ca()
residue_net = net2.Dense_rain_residual()
# ...
optimizerG = optim.Adam(netG.parameters(), ...)
# ...
optmizerG.step()

It seems that train_rain_class.py corresponds to the joint optimization stage in your paper, but only the density classifier network is updated here. Is it the expected behavior?

BTW, there is an undefined variable optimizer_D in line 156. I'm not sure what it's for.

I want to learn this code following with the readme, but there is not a demo.dy file in this repository. Would you like upload a new demo.dy?

Guidance from README.md

python derain_train_2018.py  --dataroot ./facades/DID-MDN-training/Rain_Medium/train2018new  --valDataroot ./facades/github --exp ./check --netG ./pre_trained/netG_epoch_9.pth.
Make sure you download the training sample and put in the right folder

Code Snippet in derain_train_2018.py

netG=net.Dense_rain()

if opt.netG != '':
  netG.load_state_dict(torch.load(opt.netG))
print(netG)

Why fine-tune on top of a pre-trained model?

Hello, thanks for your work first.
When I run test.py, there is an error
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/aten/src/THC/generic/THCStorage.cu:58
in
File "test.py", line 215, in
output = residue_net(val_inputv, label_cpu)

How can I handle this problem?

请问./facades/DID-MDN-training/Rain_Medium/train2018new这个文件是不是缺少了？？我没有找到。。。

关于train_rain_class.py，请问这个文件是训练label的文件吗？为什么里面的代码和论文的表述有些差异？

Could you tell me hou to solve it ?

When I lode the dataset,it raised Exception: Check dataroot.I changed the path in pix2pix_class.py and the dataroot in the command,but it still didn't work.

https://sites.google.com/site/hezhangsprinter/

Training (heavy, medium, light) and testing (TestA and Test B) data can be downloaded at the following link: https://drive.google.com/file/d/1cMXWICiblTsRl1zjN8FizF5hXOpVOJz4/view?usp=sharing

I'am sorry but I can't access the drive.google.com .

models.vgg19_bn(pretrained=True)
vgg19_bn在哪里？

Hi! I want to retrain DID-MDN using my dataset (without density-labels). I hope to only retrain multi-stream dense network instread of the whole network. Thank you very much!

hello , i have a question about reproduce ,out_image is strange by your weights, maybe there are something wrong in png. and jpg , could you describe reproduction in detail , thanks a lot

作者你好，请问获取残差的时候，为什么无论输入的图片是大中小三中雨，输入的label都是0 呢？output=residue_net(input, label_cpu)谢谢

When I run test.py (I have put the 3 pertrained models into their directories), the error came out:
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

How can I solve the problem?

Hi,Zhang! There is a problem that the structure of the "BottleneckBlock" in derain_residual.py or derain_dence.py is not reasonable. The order should be conv、relu and bn, but is opposite,in the code.
Please give a explain，thank you!

Seemingly, there is three training stage according to your paper:

• Stage 1: training residual extraction network only
• Stage 2: training density classification network based on residue output from the network in Stage 1
• Stage 3: joint training of both networks

quote: "To efficiently train the classifier, a two-stage training protocol is leveraged. A residual feature extraction network is firstly trained to estimate the residual part of the given rainy image, then a classification sub-network is trained using the estimated residual as the input and is optimized via the ground truth labels (rain-density). Finally, the two stages (feature extraction and classification) are jointly optimized."

It seems that the code of Stage 1 corresponds to file derain_residual.py, and Stage 3 corresponds to file train_rain_class.py (which directly loads two pre-trained models). How to get the density classifier trained in Stage 2?

你好，有个问题为什么batchsize设置为1啊？

I am sorry to I am not understand the code.
In derain_dense.py, I find that the 'Dense_base_down0' code is:
x11 = self.upsample(self.relu((self.conv11(x1))), size=shape_out)
x21 = self.upsample(self.relu((self.conv11(x1))), size=shape_out)
x31 = self.upsample(self.relu((self.conv11(x1))), size=shape_out)
x41 = self.upsample(self.relu((self.conv11(x1))), size=shape_out)
x51 = self.upsample(self.relu((self.conv11(x1))), size=shape_out)
why did you do the same operation?
Does it conv to x1,x2,x3,x4,x5??

cheers

when I run "python train_rain_class.py --dataroot ./facades/DID-MDN-training/Rain_Medium/train2018new --exp ./check_class" got this error. And I look into the train_rain_class.py. And I didn't find the definition of 'optimizerD'.

您好，请问您论文中ssim和psnr指标是如何计算的，rgb图像直接计算还是仅仅计算ycbcr空间的y通道？？？

Hi there.
I was following the Readme file to setup the environment, including ubuntu+python2.7 etc. And I have downloaded all pretrained models and put them in the corresponding directory. But when I tried to run the test.py file using the command python test.py --dataroot ./facades/github --valDataroot ./facades/github --netG ./pre_trained/netG_epoch_9.pth, an error occurred and said TypeError: __init__() got an unexpected keyword argument 'pretrained'
Can you or anyone help me out? Really appreciate it!

Hi, Zhang. After reading your code, I have some puzzles. In derain_dense.py, the class Dense_base_down2 has defined conv11, conv21, ... , conv61, however, in the forward function, you only use conv11. It is quite different from the description in your paper. Could you please explain it?

It seems the code can only produce a de-rain output with size of 512*512, so do I have to resize it back to the original size in order to get the PSNR and SSIM value? Will that effect the result?

你好，test1200数据集不能下载

There are only three folders in the data set provided by the paper: Rain_Heavy/train2018new, Rain_Medium/train2018new,
Rain_light/train2018new
Other folder data sets do not exist, it will affect the program running? Where is it? thank you very much!

I find the value of loss is not stable.In the paper,the loss function consist of two parts，but only one in the code.

Because I want to do a comparative experiment.Thank you very much!
my email：[email protected]

Is there a lot of error in this code? Why does it not correspond to many parts of the paper

关于train_rain_class.py，请问这个文件是训练label的文件吗？为什么里面的代码和论文的表述有些差异？