Efficient Video Super-Resolution via Hierarchical Temporal Residual Networks

By Zhi-Song Liu, Wan-Chi Siu and Yui-am Chan

The paper can be found in IEEE

   @ARTICLE{8792098,  
          author={Z. {Liu} and W. {Siu} and Y. {Chan}},  
          journal={IEEE Access},   
          title={Efficient Video Super-Resolution via Hierarchical Temporal Residual Networks},   
          year={2021},  
          volume={7},  
          number={},  
          pages={129112-129126},
   }

• We propose a Space-Time Convolutional Neural Network, in which we have improved the residual block by adding an adaptive weighting convolution process and global & local connections to learn deep feature representation for video SR.

• The convolution in ST-CNN is done by our proposed Spatial Convolution Packing (SCP) scheme. We can then use fewer parameters and less computation to combine the channel information with the spatial domain for joint training.

• To boost up the visual quality, we also combine the frame overlapping and patch overlapping processes to form ST-CNN as the proposed ST-CNN+ to further enhance the SR quality in terms of PSNR.

Python > 3.0
OpenCV library
CAFE
NVIDIA GPU + CUDA
MATLAB 6.0 or above

The complete architecture is shown as follows,

Proposed Spatial Convolution Packing (SCP) scheme.

You can download the pre-trained models from: https://drive.google.com/file/d/1N_3AYEhKCgItjdyvijzCLXLPrTVzqh2X/view?usp=sharing

For STCNN

run ST-CNN_test.ipynv

This table shows the comparison among different video SR algorithms.

This figure shows the running time comparison.

This figure shows the visual comparison.

Please cite our paper for using our dataset or models.