corochann / seranet Goto Github PK

Super Resolution of picture images using deep learning

License: MIT License

Python 100.00%

deep-convolutional-networks convolutional-neural-networks resolution super-resolution chainer deep-learning python picture

seranet's Introduction

SeRanet

Super Resolution of picture images using deep learning.

Demo

SeRanet upscales picture image size to x2.

* Below sample pictures are obtained from PEXELS, providing public domain photos.

Before upscale

Conventional method: image upscaled with lanczos method (with OpenCV library)

SeRanet: image upscaled using seranet_v1

Original image (= Ground truth data, for reference)

Description

SeRanet project aims to Study and Research how deep convolutional neural network works to learn super resolution of the image.

It is developed with python on chainer framework, flexible machine learning library.

Explanation material about super resolution through convolutional neural network is uploaded on SlideShare.

References

The project is inspired by following two reference

Chao Dong, Chen Change Loy, Kaiming He, Xiaoou Tang, "Image Super-Resolution Using Deep Convolutional Networks", http://arxiv.org/abs/1501.00092

The term "SRCNN", super resolution using deep convolutional neural network, is introduced in this paper.

waifu2x

Image super resolution for Anime-Style art and photo. You can also test the application at http://waifu2x.udp.jp/index.html.

Machine learning library

chainer

Machine learning library which can be written in python. It is open source on github

Environment setup and requirements

I tested the software with Ubuntu 14.04, python 2.7.

Running the software requires a lot of memory, more than 8GB RAM recommended (it depends on which architecture to use).

numpy
chainer

Follow github page for installation. Setting up GPU (enable CUDA, cuDNN) accelerates the calculation. It is recommended to use GPU if possible.

OpenCV

OpenCV is used for image processing. Installation page

Also, please set environment variable PYTHONPATH='.'. Add below code in ~/.bashrc.

export PYTHONPATH=.:${PYTHONPATH}

How to use

Basic usage

Just specify image file path which you want to upscale. If you get memory error or exit code 137, please try with smaller size image.

Ex. Upscaling input.jpg

python src/inference.py input.jpg

Specify output file name and path

Ex. Upscaling /path/to/input.jpg to /path/to/output.jpg

python src/inference.py /path/to/input.jpg /path/to/output.jpg

Specify model to use:

SeRanet project studies several network architecture. You can specify which network archtecture to use for SR with -a option.

Ex. use model seranet_v1

python src/inference.py /path/to/input.jpg /path/to/output.jpg -a seranet_v1

Use GPU:

GPU makes calculation much faster. Specify -g option is to use GPU.

python src/inference.py /path/to/input.jpg /path/to/output.jpg -g 0

Training

You can construct your own convolutional neural network, and train it!

1. Data preparation

Put training images[1] inside data/training_images directory. (I used 5000 photo images during the training, see next section "Training images".)

[1]: Currently, images will be cropped to size 232 x 232 during training.

2. Training the model

Once prepared training_images, see code/arch/ directory to choose which model to train, and execute below. -g 0 is to use GPU. (For the training, it is highly recommended to use GPU, otherwise training don't finish maybe a month...)

python src/train.py -a model_name -g 0

Training images

Training images may be more important than algorithm of deep learning to achieve high quality super resolution. Generally, training images dataset distributed for image classification task are not so high quality in terms of sharpness of the images.

Therefore, I chose PEXELS photos for the training of SeRanet.

PEXELS

The website provides high quality photos under Creative Commons Zero (CC0) license.

Thanks to the pexels team and photographers, I can re-distribute training images dataset which I used. It consists of 5000 medium size PEXELS photos. You can download from below,

Download PEXELS training_images (jump to google drive, file size around 269MB)

After extract, copy this to data/training_images to start your own training.

Contribution is welcome

The performance of SR for this project is still not matured. You are welcome to suggest any improvement & contribute to this project. If you could get any model which performs better performance, feel free to send me a pull request!

seranet's People

Contributors

Stargazers

Watchers

Forkers

hzy-zg amith-moorkoth jqsheng94 kazava yifanw90 suntaopku liu3xing3long duke24k xuhuapeng garymatthews

seranet's Issues

memory error

prepare model
-------- training parameter --------
GPU ID : -1
archtecture : seranet_v1
batch size : 5
epoch : 1000
color scheme : rgb
size : 64

loading data
file size 5000
total skip file size = 0
after resize: data_x.shape (5000, 3, 32, 32) sum 1890920924.0
after resize: data_y.shape (5000, 3, 64, 64) sum 7555895403.0
setup model
training
epoch: 1
Traceback (most recent call last):
File "src/train.py", line 203, in
optimizer.update(model, x, t)
File "C:\Python\lib\site-packages\chainer\optimizer.py", line 390, in update
loss = lossfun(*args, **kwds)
File "C:\Users\Amith Moorkoth\Desktop\im\src\arch\seranet_v1.py", line 87, in call
h = F.leaky_relu(self.conv11(h), slope=0.1)
File "C:\Python\lib\site-packages\chainer\links\connection\convolution_2d.py", line 108, in call
deterministic=self.deterministic)
File "C:\Python\lib\site-packages\chainer\functions\connection\convolution_2d.py", line 326, in convolution_2d
return func(x, W, b)
File "C:\Python\lib\site-packages\chainer\function.py", line 199, in call
outputs = self.forward(in_data)
File "C:\Python\lib\site-packages\chainer\function.py", line 312, in forward
return self.forward_cpu(inputs)
File "C:\Python\lib\site-packages\chainer\functions\connection\convolution_2d.py", line 69, in forward_cpu
cover_all=self.cover_all)
File "C:\Python\lib\site-packages\chainer\utils\conv.py", line 33, in im2col_cpu
col = numpy.ndarray((n, c, kh, kw, out_h, out_w), dtype=img.dtype)
MemoryError

Error on using inference.py

Hi
thanks for the implementation of Image SRCNN
After installation as mentioned in readme.txt, when I run inference.py with input and output image arguments on a GPU, I get the following error

(keras) tharun@nvidia-All-Series:~/keras/SeRanet$ python src/inference.py /home/tharun/Desktop/ID099.jpg /home/tharun/Desktop/11.jpg -g 1
prepare model
Traceback (most recent call last):
File "src/inference.py", line 92, in
model = model_arch.seranet_v1(inout_ch=inout_ch)
File "/home/tharun/keras/SeRanet/src/arch/seranet_v1.py", line 27, in init
seranet_v1_crbm=seranet_v1_crbm(inout_ch),
File "/home/tharun/keras/SeRanet/src/arch/seranet_v1.py", line 130, in init
crbm1=ConvolutionRBM(inout_ch, 64, 5, wscale=1.),
File "/home/tharun/keras/SeRanet/src/arch/convolution_rbm.py", line 51, in init
conv=L.Convolution2D(in_channels, out_channels, ksize, stride=stride, wscale=wscale),
File "/home/tharun/keras/local/lib/python2.7/site-packages/chainer/links/connection/convolution_2d.py", line 112, in init
argument.assert_kwargs_empty(kwargs)
File "/home/tharun/keras/local/lib/python2.7/site-packages/chainer/utils/argument.py", line 18, in assert_kwargs_empty
parse_kwargs(kwargs)
File "/home/tharun/keras/local/lib/python2.7/site-packages/chainer/utils/argument.py", line 12, in parse_kwargs
raise TypeError('got unexpected keyword argument(s) %s' % args)
TypeError: got unexpected keyword argument(s) 'wscale'

Hope to get it resolved ..
Thanks in advance

Best
Tharun

No module named src.functions

I use python2

$ python src/inference.py image.png 
prepare model
Traceback (most recent call last):
  File "src/inference.py", line 91, in <module>
    import arch.seranet_v1 as model_arch
  File "/home/user/SeRanet/src/arch/seranet_v1.py", line 8, in <module>
    import src.functions as CF
ImportError: No module named src.functions

Issue in importing ConvolutionRBM

prepare model
Traceback (most recent call last):
File "src/inference.py", line 91, in
import arch.seranet_v1 as model_arch
File "C:\Users\amith\Desktop\im\src\arch\seranet_v1.py", line 10, in
from convolution_rbm import ConvolutionRBM
ImportError: No module named 'convolution_rbm'

when try to execute on windows

Single 96*96 image on titanx gpu takes longtime

I have successfully installed and tried to run inference.py using basic_cnn_small model on a single GPU titanX, but it takes more than 3 hours and still going on, kindly help
(keras) tharun@nvidia-All-Series:~/keras/SeRanet$ python
Python 2.7.6 (default, Oct 26 2016, 20:30:19)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import chainer
chainer.cuda.available
True
chainer.cuda.cudnn_enabled
True

(keras) tharun@nvidia-All-Series:~/keras/SeRanet$ python src/inference.py /media/data_raid/tharun/noyawn/faces/0.png /media/data_raid/tharun/noyawn/01.png -g 2 -a basic_cnn_small
prepare model
setup model
loading data
upscaling to /media/data_raid/tharun/noyawn/01.png

Regards
Tharun

batch process?

Hello ! everything is working, CUDA on GTX1080, I'd just really love to know how to process an entire folder of images ?
Sorry this is maybe a noob question.
That would be amazing !

Many thanks

edit : nevermind, I found a way to do with shell. If anyone is interested, just create a .sh file in the folder , and put this inside :

"
for file in images/*.png
do
python src/inference.py -g 0 $file
done
"

also create a folder called "images" with your pictures in it

ModuleNotFoundError: No module named 'src'

I want to do network learning.
・Create data / Training_images
・Place my data set in Training_images
・Run 　python src / train.py -a seranet_v1 -g 0
I attempted learning by the above procedure.
However, I got an error.
ModuleNotFoundError: No module named 'src'
I studied variously and tried changing it myself, but I could not solve it