Git Product home page Git Product logo

hbpn's Introduction

PWC

PWC

PWC

HBPN (Hierarchical-Back-Projection-Network-for-image-SR)

By Zhi-Song, Li-Wen Wang and Chu-Tak Li

This repo only provides simple testing codes, pretrained models and the network strategy demo.

We propose a single image super-resolution using Hierarchical Back Projection Network (HBPN) to achieve good SR performance with low distortion.You can check the paper from arxiv

BibTex

    @InProceedings{Liu2019hbpn,
        author = {Liu, Zhi-Song and Wang, Li-Wen and Li, Chu-Tak and Siu, Wan-Chi},
        title = {Hierarchical Back Projection Network for Image Super-Resolution},
        booktitle = {The Conference on Computer Vision and Pattern Recognition Workshop(CVPRW)},
        month = {June},
        year = {2019}
    }

For proposed HBPN model, we claim the following points:

• Enhanced back projection blocks.

• Adopt the HourGlass structure in back projection to gradually explore deeper and denser feature representation.

• Softmax based Weighted Reconstruction (WR) block to hierarchically minimize residues between SR and HR images.

Dependencies

Python 2.XXX<3.0
OpenCV liberary
Caffe 
NVIDIA GPU + CUDA
Jupyter Notebook
MATLAB 6.0 and above

Complete Architecture

The complete architecture is shown as follows,

structure

Implementation

1. Testing


s1. Download trained HBPN model from the following link:

for NTIRE2019 testing dataset, download the model from the following link:

https://drive.google.com/open?id=1AeUiztLk0mrY8VwRtr4BVfnshNIM20xa

for 2x, 4x and 8x general testing dataset, download the model from the following link:

https://drive.google.com/open?id=1vcY7C-O87s9fWPlxNeEkHFCBoS8Z634T

s2. Run HBPN_main.ipynb on Jupyter Notebook. Modify the directories of files based on your working environment.

Testing images on NTIRE2019 Real Super-Resolution Challenge can be downloaded from the following link:

https://drive.google.com/open?id=1OnVRlOM6mS6Zk2QvXpa9wJ0lfV3CH3-4

General testing dataset (Set5, Set14, BSD100, Urban100 and Manga109) can be downloaded from:

https://github.com/LimBee/NTIRE2017

2. Training


s1. Download the training images from DIV2K and Flickr.

https://data.vision.ee.ethz.ch/cvl/DIV2K/

https://github.com/LimBee/NTIRE2017

s2. Generate training/testing files

Find data generation code in Matlab and run patch_collection.m to generate training/testing HDF5 files and put them in Data folder.
Find train.txt and test.txt in code folder and change the directories of generated HDF5 files.

s3. Start training on Caffe

For user who already has installed Caffe, simply just run the following code:

$ caffe train -solver HBPN_solver.protxt -GPU=0,1 2>&1 | tee -a HBPN.log

You can also use docker to install Caffe to run the code:

$ docker pull bvlc/caffe:gpu
$ nvidia-docker run -it --name [container_name] -p 8888:8888 -v ~/Data:/Data [image_name]

For testing, to call Jupyter for running:

$ jupyter notebook --ip=0.0.0.0 --port=8888 --allow-root &

Experimental results

All the testing results on Set5, Set14, BSD100, Urban100 and Manga109 can be downloaded from the following link:

https://drive.google.com/open?id=1esG6op8BePCEYsL43Fx8uXo4kmZGElbX

Testing results on NITRE2019 RealSR can be downloaded from the following link:

https://drive.google.com/open?id=1ASK7U3XU8zi6W5wbW8osZwgJ9jE04r1C

Partial image visual comparison

visual compare

Complete image visual comparison

img_031.png in Urban100 dataset

Urban100 visual compare

302008.png in BSD100 dataset

BSD100 visual compare

Quantitative Comparison

quantitative compare

hbpn's People

Contributors

holmes-alan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

hbpn's Issues

questions about the paper

Hi,
Just want to make sure if there's typo in the paper.
Figure 3. describe the structure of UBP and DBP.
Beside the Conv op in UBP states 2Mx2NxC, which is quit confusing for me.
As far as I have understood, this Conv downsample the Deconv(x) and would be subtracted with x to construct residual map.
Shouldn't the shape be MxNx2C to fit the shape of x?
The same situation happens in DBP's Deconv.

Impressive paper with strong results.

Hi, I recently noticed your repository and read your paper. Nice try!

Meanwhile, I have some minor questions about the results:
a) The PSNR results are much higher than SOTA on Urban100, but relatively less impressive on others. Are there any particular reasons?
b) The SOTA results on benchmark datasets seem to be extremely inconsistent with HBPN RealSR (NTIRE2019) challenge rank. However, some NTIRE19 leading methods still report SOTA results on popular datasets. Which means better RealSR methods lead to higher PSNR on benchmark datasets, conversely, it cant be held?

Thanks in advance, and willing to know your opinions.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.