Git Product home page Git Product logo

davanet's Introduction

DAVANet

Code repo for the paper "DAVANet: Stereo Deblurring with View Aggregation" (CVPR'19, Oral).  [Paper]   [Project Page] 

Stereo Blur Dataset

Download the dataset (192.5GB, unzipped 202.2GB) from [Data Website].

Pretrained Models

You could download the pretrained model (34.8MB) of DAVANet from [Here].

(Note that the model does not need to unzip, just load it directly.)

Prerequisites

  • Linux (tested on Ubuntu 14.04/16.04)
  • Python 2.7+
  • Pytorch 0.4.1
  • easydict
  • tensorboardX
  • pyexr

Installation

pip install -r requirements.txt

Get Started

Use the following command to train the neural network:

python runner.py 
        --phase 'train'\
        --data [dataset path]\
        --out [output path]

Use the following command to test the neural network:

python runner.py \
        --phase 'test'\
        --weights './ckpt/best-ckpt.pth.tar'\
        --data [dataset path]\
        --out [output path]

Use the following command to resume training the neural network:

python runner.py 
        --phase 'resume'\
        --weights './ckpt/best-ckpt.pth.tar'\
        --data [dataset path]\
        --out [output path]

You can also use the following simple command, with changing the settings in config.py:

python runner.py

Results on the testing dataset

Citation

If you find DAVANet, or Stereo Blur Dataset useful in your research, please consider citing:

@inproceedings{zhou2019davanet,
  title={{DAVANet}: Stereo Deblurring with View Aggregation},
  author={Zhou, Shangchen and Zhang, Jiawei and Zuo, Wangmeng and Xie, Haozhe and Pan, Jinshan and Ren, Jimmy},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year={2019}
}

Contact

We are glad to hear if you have any suggestions and questions.

Please send email to [email protected]

Reference

[1] Zhe Hu, Li Xu, and Ming-Hsuan Yang. Joint depth estimation and camera shake removal from single blurry image. In CVPR, 2014.

[2] Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In CVPR, 2017.

[3] Orest Kupyn, Volodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Jiri Matas. Deblurgan: Blind motion deblurring using conditional adversarial networks. In CVPR, 2018.

[4] Jiawei Zhang, Jinshan Pan, Jimmy Ren, Yibing Song, Lin- chao Bao, Rynson WH Lau, and Ming-Hsuan Yang. Dynamic scene deblurring using spatially variant recurrent neural networks. In CVPR, 2018.

[5] Xin Tao, Hongyun Gao, Xiaoyong Shen, Jue Wang, and Jiaya Jia. Scale-recurrent network for deep image deblurring. In CVPR, 2018.

License

This project is open sourced under MIT license.

davanet's People

Contributors

hzxie avatar sczhou avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

davanet's Issues

Pretrained model of DeblurNet

Dear Shangchen Zhou,
for my research scope, I needed to infer the results produced by the single DeblurNet. I followed the procedure described (here) and was able to switch to do the Single Deblurnet configuration. I shall be grateful if you kindly help me to get the checkpoint of the single deblurnet. Specifically, I am looking for the pre-trained model which is referred as "Ours-Single" in table 1 of the paper.

codebert

Thanking you,
Oaishi.

Training iterations & FlyingThings3D

Dear Zhou:
I run the training code and find that the epoch num is 400 in config.py, while 2000k iterations is proposed for the deblurNet. Are 400 epochs sufficient for training deblurNet?
Another issue is how to train with FlyingThings3D is not presented clearly in code, thank you for more clear guidance~

The pretrained model

Hi DAVANet Team,

Can you please share the pre-trained model so we can test on it

one question about build.py

in build.py line124

deblurnet_lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(dispnet_solver,milestones=cfg.TRAIN.DEBLURNET_LR_MILESTONES,gamma=cfg.TRAIN.LEARNING_RATE_DECAY)

why 'dispnet_solver' appears here....
thanks for much

ZED camera calibration

Hello, thank you so much for the great repository. I wanted to know about the procedure of your dataset preparation. Did you use the default calibration settings of ZED camera? Or redefined the calibration yourself?

Trying to run inference using deblurnet on GOPRO dataset -- again

Hi I am following up the question I asked earlier in here:

#6

But I am specifying 'DeblurNet' instead of 'StereoDeblurNet' in the configuration file, and still the network complains about the tensor size mismatch. As from your Table1, it feels you have two different models for deblurring; one using the fuse information from fusionNet and DispBiNet while other not using this information. Is the trained model doing mono deblurring available?

Also, can you please not close an issue as someone could have follow up questions.

Memory-Usage and Performance

Hello, your work is really amazing!

I have some questions.On Gopro datasets, I used a single deblurnet(yours)to train the parameters. The experimental setup is that the train_batch_size is 2, the test_bs is 1, the initial lr is 1e-4, and lr is decayed by 0.1 in [80, 160, 240], and the other parameters are consistent with your code.

I have a few problems :
(1) Such a small batch, Memory-Usage and GPU-Util are small, is this a waste of resources? I ran 200 epochs in a day and a half, and the PSNR only reached 28.8 not 30.55 reported in your paper, but there is no sign of rising.
(2) If I use a large batch_size, the PSNR performance is worse.

What do I need to pay attention to in terms of training speed and code tricks to reproduce the results of around 30.5 in your paper? If you can give some guidance, I really appreciate you!

Thanks for your nice work again!

some questions about the test result

when I set the phase to test,I get the following result,I could not find the deblur result,and the deblured picture's color strange,what should I do?
result

left picture is original picture,the middle picture is original sharp picture,and the right picture is deblured picture use your given model{ckpt-best.pth.zip}

作者您好我想问一下torch版本的问题

我这里用的torch版本高于1.1.0,所以报出了warning。我想根据warning去调整lr_scheduler.step()和optimizer.step()的位置,但是我一直找不到optimizer.step()的位置

How to run the inference on stereo image pairs in the wild?

hi
thanks a lot for the great project, looks amazing.

Can you help on how to run the inference on images in the wild with the pre-trained model?
Do I need to modify the test code to do so or is there some code to do that already in the repo?
What data do I need to give as an input to the model, just the left/right image pairs or do I need to pass a depth map as well?

KeyError: 'epoch_idx' while testing the `deblurnet` mode

Hello,

Thank you for open-sourcing the codebase. I was trying to test your algorithm with the option deblurnet and I face the following error :

Traceback (most recent call last):
  File "runner.py", line 56, in <module>
    main()
  File "runner.py", line 53, in main
    bulid_net(cfg)
  File "/home/DAVANet/core/build.py", line 106, in bulid_net
    init_epoch = checkpoint['epoch_idx']+1
KeyError: 'epoch_idx'

Can you help fix this? Much appreciated !

Thanks for your work,I have some questions

1)how could I train gopro dataset?
2)now,I am working with car card deblur,does your work is fit in my project?
3)how about the GPU memory usage?and the test speed in gopro dataset?
Thanks

作者您好。关于你开源的代码的一些问题

您好,在原论文中,DeblurNet第13-14的子块是AtrousResBlock,但是在开源的代码中使用的ResBlock,是不是开源时代码不小心错误?
如果黄色子块确实是AtrousResBlock而不是ResBlock,可以给出AtrousResBlock的结果吗?因为您在论文中没有描述AtrousResBlock的结构
如果能够解答,真心感谢!

Trying to run inference using deblurnet on GOPRO dataset

Hello, I am trying to run the inference using the deblurnet module of DAVANet. Have changed the flags appropriately in the config file. But, when it goes to build_net(cfg) -- it gives me the following error:

"size mismatch for module.upconv3_i.0.weight: copying a param with shape torch.Size([128, 288, 3, 3]) from checkpoint, the shape in current model is torch.Size([128, 128, 3, 3])."

It is somehow complaining about the mismatch in the tensor sizes but I am not sure why this could be. Please advise!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.