Patch Based Image Warping for Content Aware Retargeting

A C++ implementation of Patch Based Image Warping for Content Aware Retargeting (2013), including the segmentation algorithm (Efficient Graph-Based Image Segmentation(2004)) and saliency map detection algorithm (Context-aware saliency detection(2010)) described in paper.

Instead of relying on third-party image libraries such as openCV, you only need to install this tensor library -- Eigen3. I wrote the operations required to implement these papers myself,the goal is to keep it less third-party library dependency and clean.

I wrote a cuda kernel for computing saliency to speed up the computation (from 2 min to 2 seconds), this is one of the improvements I made compared to the original implementation.

cd Image-Retargeting
# ----- Default compile setting -----
make
# ----- Or compile with CUDA support -----
make CUDA=1
# ----- Execute with defualt parameters-----
./build/default/default/patch_based_resizing

(see below for detail execution arguments)

In order to easily adjust the relevant parameters of the program, the user can adjust the parameters according to the following format through the argument list.

All parameters have default values, once a parameter is ignored, the default value is used, see src/retargeting.cpp to learn more about parameter usage and its default values.

./build/default/default/patch_based_resizing \
  --InputImage ./datasets/butterfly.png \
  --Sigma 0.5 \
  --SegmentK 500.0 \
  --MinSize 100 \
  --MergePercent 0.0001 \
  --MergeColorDist 20.0 \
  --SaveSegment true \
  --DistC 3 \
  --SimilarK 64 \
  --NumScale 3 \
  --PatchSize 7 \
  --SaveSaliency true \
  --SaveScaledSaliency true \
  --newH 300 \
  --newW 200 \
  --Alpha 0.8 \
  --QuadSize 10 \
  --WeightDST 3.0 \
  --WeightDLT 1.2 \
  --WeightDOR 3.0

• CMake

• GTest (optional ! only needed when you wish to run test suite)

• Eigen3

• Compliant C++17 compiler

  • The library is sytematically tested on following compilers

    Compiler	Version
    GCC	9.3.0
    clang	13.0.0

• CUDA toolkit (optional but highly recommend, otherwise it will take ~2min to generate saliance map using solely multithread)

For image processing we only requires Eigen3 to be installed on your system.

git clone -b '3.4' --single-branch --depth 1 https://gitlab.com/libeigen/eigen.git
cd eigen
mkdir build  
cd build 
cmake .. 
make install

Make sure Eigen3 can be found by your build system.

Download and install the CUDA Toolkit (11.4 on my computer) for your corresponding platform. For system requirements and installation instructions of cuda toolkit, please refer to the Linux Installation Guide, and the Windows Installation Guide.

Make sure the environment variable CUDA_PATH is set to the CUDA Toolkit install directory.

Also Make sure NVCC and cuda-toolkit can be found by your build system.

You need to additionally install GTest to run test-suite.

apt-get install libgtest-dev -y

Make sure GTest can be found by your build system.

Use the following commands from the project's top-most directory to run the test suite.

cd Image-Retargeting
make test
make run-test

original-grid	segmentation	significance	saliency	200 x 500	300 x 200

Thank zyu-tien for helping me debugging and giving me helpful advices while developing this project.

All the images used in the project could be downloaded from https://people.csail.mit.edu/mrub/retargetme/download.html.