@NVFX This is really awesome! Thanks for your interests and efforts. I try your demo4k.sh with another style example. The transfer result (3.5K*1.5K, takes 4.5 minutes) is shown below:

My principle of selecting the content and style is that before running the algorithm, I could somehow imagine what the result will be like in mind (e.g., spring to winter, day to night).

Just want to double check with you. It looks that such an interleave strategy is still at the cost of sacrificing the resolution a little bit, right? When I zoom in the result, I found that details on the mountain and grass are not as clear as those in the content.

The result:

The content:

The style:

from fastphotostyle.

@wesleyw72 NvBlit interleaves the content image in the horizontal and vertical direction for inference and performs the inverse when all tiles are processed. If you run the script you should see this in the content and results folder. The smoothing step is performed at the end.

NvBlit works with just about any image processing algorithm including video, texture synthesis, etc.

Please test the app & send me your feedback. I can certainly create a python library to manage GPU resources. I will then add this work to my schedule (if approved).

from fastphotostyle.

A quick test this morning...

+

from fastphotostyle.

@NVFX do tell..

from fastphotostyle.

It is possible to scale the image across multiple passes (or GPUs in parallel) to generate an HD style transfer without changing your code or script in any significant way. I don't have an official role on this project, but I am very interested in this problem.

from fastphotostyle.

I'm facing the same issue. The memory isn't cleared when the function ends, somehow it accumulates and crashes the program.

from fastphotostyle.

Same issue here. Amazing demo/sw, but I would really love to try out this code on at least 4k UHD resolution images (3840x2160). But like others have noted, GeForce 1080 runs out of memory somewhere around 800x600 resolution.

from fastphotostyle.

I can fix this issue.

from fastphotostyle.

@moeedkundi If you extract this code locally and run demo4k.sh in your FastPhotoStyle directory. It will execute the following command:

./NvBlit -w 3840 -l 2160 -i 700 -s ./images/style1.png -c $content_image -o $output_image -x "python demo4k.py --style_image_path @style_image --content_image_path @content_image --output_image_path @output_image"

...Which will generate a 4k version of the demo.sh content.

fps4k.zip

from fastphotostyle.

@NVFX What is NvBlit?

from fastphotostyle.

@wesleyw72 It is an Ubuntu 16.04 c++ application that converts your content image into a set of interleaved tiles. The tiles are of -i 700 size in this case, but it is the largest square content image you can send your GPU. The output is automatically fed to your neural network and merged together as the final pass. The code can be easily modified to send each GPU (locally or remote) a copy of the image for processing in parallel.

There is no python version at this time and the square tile size is a simplification.

demo4k.sh downloads the content EXCEPT the following lines are commented out to work with high resolution images:

do not resize for NvBlit

#convert -resize 25% content1.png content1.png;
#convert -resize 50% style1.png style1.png;

from fastphotostyle.

@NVFX Awesome! Is the source code available for this tool? Mainly interested in the technique used for merging the images. Is it a simple merge or does it do something clever?

from fastphotostyle.

@NVFX This is great work. Do you know of any resources where I could learn more about the interleaving technique?

from fastphotostyle.

@wesleyw72 @Yijunmaverick There is a final change the requires modification of the style transfer algorithm. I will discuss this information with my manager and perhaps I can get involved to fix it.

To increase resolution using NvBlit interleaving algorithm:

1. Adding noise to the ease the resolution.
2. Dithering interleaved cells based on image complexity.

Disclosing the algorithm and source is the next step in a c++ & python library.

from fastphotostyle.

@NVFX Thanks. Look forward! I could get the big picture of how such an interleave strategy works. Interested in details.

from fastphotostyle.