wwlcape / stan Goto Github PK

This is an official implementation of Video Super-Resolution via a Spatio-Temporal Alignment Network.

Python 84.82% Shell 0.89% C++ 1.85% C 0.88% Cuda 11.56%

super-resolution vsr video-super-resolution dynamic tip2022

stan's Introduction

STAN

PyTorch code for our TIP 2022 paper “Video Super-Resolution via a Spatio-Temporal Alignment Network” Paper or Researchgate.

Overview

Train

Prerequisites

CUDA 10.2/gcc 5.4/PyTorch 1.4

Train a new model:

bash install.sh
python train.py -datasets_tasks W3_D1_C1_I1

Test

lists/train_tasks_W3_D1_C1_I1.txt specifies the dataset-task pairs for training and testing.

Test a model:

python test.py -method model_name -epoch N -dataset REDS4 -task SR_color/super-resolution

Pretrained Models

The pretrained model for STAN can be found from here.

Citation

If you find the code and paper useful in your research, please cite:

@article{wen2022video,
  title={Video Super-Resolution via a Spatio-Temporal Alignment Network},
  author={Wen, Weilei and Ren, Wenqi and Shi, Yinghuan and Nie, Yunfeng and Zhang, Jingang and Cao, Xiaochun},
  journal={IEEE Transactions on Image Processing},
  volume={31},
  pages={1761--1773},
  year={2022},
  publisher={IEEE}
}

Acknowledgement

This project is based on [Learning Blind Video Temporal Consistency] and our filter adaptive alignment module is based on[STFAN].

@inproceedings{zhou2019stfan,
  title={Spatio-Temporal Filter Adaptive Network for Video Deblurring},
  author={Zhou, Shangchen and Zhang, Jiawei and Pan, Jinshan and Xie, Haozhe and  Zuo, Wangmeng and Ren, Jimmy},
  booktitle={Proceedings of the IEEE International Conference on Computer Vision},
  year={2019}
}

@inproceedings{Lai-ECCV-2018,
    author    = {Lai, Wei-Sheng and Huang, Jia-Bin and Wang, Oliver and Shechtman, Eli and Yumer, Ersin and Yang, Ming-Hsuan}, 
    title     = {Learning Blind Video Temporal Consistency}, 
    booktitle = {European Conference on Computer Vision},
    year      = {2018}
}

stan's People

Contributors

Stargazers

Watchers

Forkers

zhanglei826-stack wwhio rjmf2

stan's Issues

When we install environment, it raises this problem.

(STAN) zqiang@server-03:~/STAN-master$ bash install.sh
running clean
removing 'build/temp.linux-x86_64-cpython-37' (and everything under it)
running install
/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
setuptools.SetuptoolsDeprecationWarning,
/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/command/easy_install.py:147: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
EasyInstallDeprecationWarning,
running bdist_egg
running egg_info
writing kernelconv2d_cuda.egg-info/PKG-INFO
writing dependency_links to kernelconv2d_cuda.egg-info/dependency_links.txt
writing top-level names to kernelconv2d_cuda.egg-info/top_level.txt
reading manifest file 'kernelconv2d_cuda.egg-info/SOURCES.txt'
writing manifest file 'kernelconv2d_cuda.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
building 'kernelconv2d_cuda' extension
creating /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/build/temp.linux-x86_64-cpython-37
Emitting ninja build file /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/build/temp.linux-x86_64-cpython-37/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/2] :/usr/local/cuda/bin/nvcc -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/TH -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/THC -I:/usr/local/cuda/include -I/share4/home/zqiang/anaconda3/envs/STAN/include/python3.7m -c -c /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/KernelConv2D_kernel.cu -o /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/build/temp.linux-x86_64-cpython-37/KernelConv2D_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=kernelconv2d_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
FAILED: /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/build/temp.linux-x86_64-cpython-37/KernelConv2D_kernel.o
:/usr/local/cuda/bin/nvcc -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/TH -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/THC -I:/usr/local/cuda/include -I/share4/home/zqiang/anaconda3/envs/STAN/include/python3.7m -c -c /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/KernelConv2D_kernel.cu -o /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/build/temp.linux-x86_64-cpython-37/KernelConv2D_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=kernelconv2d_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
/bin/sh: 1: :/usr/local/cuda/bin/nvcc: not found
[2/2] c++ -MMD -MF /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/build/temp.linux-x86_64-cpython-37/KernelConv2D_cuda.o.d -pthread -B /share4/home/zqiang/anaconda3/envs/STAN/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/TH -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/THC -I:/usr/local/cuda/include -I/share4/home/zqiang/anaconda3/envs/STAN/include/python3.7m -c -c /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/KernelConv2D_cuda.cpp -o /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/build/temp.linux-x86_64-cpython-37/KernelConv2D_cuda.o -std=c++11 -std=c++14 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=kernelconv2d_cuda -D_GLIBCXX_USE_CXX11_ABI=0
FAILED: /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/build/temp.linux-x86_64-cpython-37/KernelConv2D_cuda.o
c++ -MMD -MF /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/build/temp.linux-x86_64-cpython-37/KernelConv2D_cuda.o.d -pthread -B /share4/home/zqiang/anaconda3/envs/STAN/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/TH -I/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/THC -I:/usr/local/cuda/include -I/share4/home/zqiang/anaconda3/envs/STAN/include/python3.7m -c -c /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/KernelConv2D_cuda.cpp -o /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/build/temp.linux-x86_64-cpython-37/KernelConv2D_cuda.o -std=c++11 -std=c++14 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=kernelconv2d_cuda -D_GLIBCXX_USE_CXX11_ABI=0
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /share4/home/zqiang/STAN-master/networks/FAC/kernelconv2d/KernelConv2D_cuda.cpp:3:0:
/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/include/ATen/cuda/CUDAContext.h:5:30: fatal error: cuda_runtime_api.h: No such file or directory
compilation terminated.
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1400, in _run_ninja_build
check=True)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "setup.py", line 28, in
'build_ext': BuildExtension
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/init.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
self.run_command(cmd)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/command/install.py", line 74, in run
self.do_egg_install()
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/command/install.py", line 123, in do_egg_install
self.run_command('bdist_egg')
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
self.distribution.run_command(command)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 165, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
self.run_command(cmdname)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
self.distribution.run_command(command)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
self.run_command('build_ext')
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
self.distribution.run_command(command)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 84, in run
_build_ext.run(self)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
self.build_extensions()
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 580, in build_extensions
build_ext.build_extensions(self)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
self._build_extensions_serial()
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
self.build_extension(ext)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
_build_ext.build_extension(self, ext)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 554, in build_extension
depends=ext.depends,
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 423, in unix_wrap_ninja_compile
with_cuda=with_cuda)
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1140, in _write_ninja_file_and_compile_objects
error_prefix='Error compiling objects for extension')
File "/share4/home/zqiang/anaconda3/envs/STAN/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1413, in _run_ninja_build
raise RuntimeError(message)
RuntimeError: Error compiling objects for extension

about train dataset

https://drive.google.com/file/d/1vV-HIyKnVrNa8h2LVyRezleVmbwkVBy3/view?usp=sharing
Please，what dataset is used for this link?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.