The above code is an interface inference in c++ against a Faster R-CNN trained network.

From the code developed by Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun (Microsoft Research) for the project py-faster-rcnn, some modules and files from the py-faster-rcnn/lib and py-faster-rcnn/data/scripts/ folders are used directly:

• RPN module (py-faster-rcnn/lib/rpn/): Needed to deploy the region proposal network.
• NMS module (py-faster-rcnn/lib/nms/): Needed to apply non-maximum suppression step.
• Fast_rcnn module (py-faster-rcnn/lib/fast_rcnn/): Contains auxiliary functions.
• py-faster-rcnn/lib/Makefile and py-faster-rcnn/lib/setup.py: To compile NMS CUDA and Cython libraries.
• py-faster-rcnn/data/scripts/fetch_faster_rcnn_models.sh: To download pre-computed Faster R-CNN detectors.

This code is added as a submodule to the present project for convenience. It also uses their branch of the framework Caffe (caffe-fast-rcnn).

FASTER_RCNN C++ class has adapated (copy/modify/correct) some fuctions from the project FasterRCNN-Encapsulation-Cplusplus. Thanks so much to his SourceCode.

First of all, clone this project with --recursive flag:

git clone --recursive https://github.com/HermannHesse/faster_rcnn_cplusplus.git

1. Requirements for Caffe and pycaffe (see: Caffe installation instructions)

   Note: Caffe must be built with support for Python layers!

   # In your Makefile.config, make sure to have this line uncommented
   WITH_PYTHON_LAYER := 1
   # Unrelatedly, it's also recommended that you use CUDNN
   USE_CUDNN := 1

   You can download Ross Girshick Makefile.config for reference.

2. Python packages you might not have: cython, python-opencv, easydict

3. Build Caffe and pycaffe

   cd $ROOT_DIR/py-faster-rcnn/caffe-fast-rcnn/
   # Now follow the Caffe installation instructions here:
   #   http://caffe.berkeleyvision.org/installation.html
   
   # If you're experienced with Caffe and have all of the requirements installed
   # and your Makefile.config in place, then simply do:
   make -j8 && make pycaffe

4. Build the Cython modules

   cd $ROOT_DIR/py-faster-rcnn/lib/
   make

5. Download pre-computed Faster R-CNN detectors

   cd $ROOT_DIR/py-faster-rcnn/
   ./data/scripts/fetch_faster_rcnn_models.sh

1. Set $ROOT_DIR/py-faster-rcnn/caffe-fast-rcnn/python/ and $ROOT_DIR/py-faster-rcnn/lib/ into the enviroment variable PYTHONPATH

   export PYTHONPATH=$ROOT_DIR/py-faster-rcnn/caffe-fast-rcnn/python/:$ROOT_DIR/py-faster-rcnn/lib/:${PYTHONPATH}

2. Compile the project with CMake

   mkdir $ROOT_DIR/build/
   cd $ROOT_DIR/build/
   cmake ..
   make
   mv faster_rcnn_cplusplus ..

3. Run the demo

   cd $ROOT_DIR
   ./faster_rcnn_cplusplus

"Unknown layer type: Python"

Caffe has been probably compiled without PYTHON_LAYER support. Uncomment the following line

WITH_PYTHON_LAYER := 1

in $ROOT_DIR/py-faster-rcnn/caffe-fast-rcnn/Makefile.config file and recompile it.

cd $ROOT_DIR/py-faster-rcnn/caffe-fast-rcnn/
make clean
make -j8 && make pycaffe

fatal error: caffe/proto/caffe.pb.h: No such file or directory
#include "caffe/proto/caffe.pb.h"
                                  ^
compilation terminated.

caffe.pb.h is a header file generated by Google Protocol Buffer and it is missing for some reason. Let's create it.

cd $ROOT_DIR/py-faster-rcnn/caffe-fast-rcnn/
protoc src/caffe/proto/caffe.proto --cpp_out=.
mkdir include/caffe/proto
mv src/caffe/proto/caffe.pb.h include/caffe/proto

Reference full discussion from here.

When the net is loading Caffe can't find rpn python layer:

ImportError: No module named rpn.proposal_layer

Add lib/ directory to PYTHONPATH:

export PYTHONPATH=$ROOT_DIR/py-faster-rcnn/lib/:${PYTHONPATH} 

Python can't find gpu_nms library:

from nms.gpu_nms import gpu_nms
ImportError: No module named gpu_nms

Must compile libraries gpu_nms.so and cpu_nms.sodone by running the fourth step:

cd $ROOT_DIR/py-faster-rcnn/lib/
make

/usr/bin/ld: cannot find -lopencv_dep_cudart 
collect2: error: ld returned 1 exit status

There is a problem with the CMakeCache.txt and the enviroment variable CUDA_USE_STATIC_CUDA_RUNTIME. To solve this issue, set it OFF in CMakeList.txt:

# Add the following line to CMakeList.txt and recompile
set(CUDA_USE_STATIC_CUDA_RUNTIME "OFF")

or generate CMake files with:

cd $ROOT_DIR/build/
cmake .. -D CUDA_USE_STATIC_CUDA_RUNTIME=OFF

Reference full discussion from here.