Code for my master thesis, titled "Real-Time Multiple Object Tracking: A Study on the Importance of Speed".

In this project, we implement a multiple object tracker, following the tracking-by-detection paradigm, as an extension of an existing method. It works by modelling the movement of objects by solving the filtering problem, and associating detections with predicted new locations in new frames using the Hungarian algorithm. Three different similarity measures are used, which use the location and shape of the bounding boxes. Compared to other trackers on the MOTChallenge leaderboard, our method, referred to as C++SORT, is the fastest non-anonymous submission, while also achieving decent score on other metrics. By running our model on the Okutama-Action dataset, sampled at different frame-rates, we show that the performance is greatly reduced when running the model - including detecting objects - in real-time. In most metrics, the score is reduced by 50%, but in certain cases as much as 90%. We argue that this indicates that other, slower methods could not be used for tracking in real-time, but that more research is required specifically on this.

This work constitutes my master thesis. The thesis is published on a university portal, but is also made avalable on arXiv.

If you find the thesis or this repository useful in your research, please consider citing the report:

@article{murray2017real,
  title={Real-Time Multiple Object Tracking - A Study on the Importance of Speed},
  author={Murray, Samuel},
  journal={arXiv preprint arXiv:1709.03572},
  year = {2017}
}

• Build:
  • C++11
  • make
  • pkg-config
• Tracking:
  • dlib (≥19.4)
• Detection:
  • OpenCV (≥3.2)
  • Caffe^*
  • Cuda^* (≥7)
• Run demos:
  • Boost (≥1.63)

* For instructions on how to install Caffe and Cuda, see Configure.

Use Homebrew to build packages.

To install OpenCV, follow this tutorial.

Remaining dependencies can be installed with:

$ brew install <package>

It might be possible to build all packages yourself, without Homebrew. Please refer to the instructions for Linux below.

To install boost, see the official documentation.

To install OpenCV, see the official documentation.

To install dlib, download and unpack it from here. Build as a shared library with:

$ cd dlib-<version>
$ mkdir build
$ cd build
$ cmake ..
$ make
$ sudo make install

Not tested. Please refer to the instructions for Linux above; some packages have similar guides for Windows.

The following directories should be present in root:

• config - .pc files for dependencies.
• cpp - C++ source code.
• data - Data sets and sequence maps for running demos.
• models - Caffe models for object detection.
• python - Python scripts for generating images.

Make sure PKG_CONFIG_PATH includes the paths to the .pc files for all dependencies. Alternatively, copy and modify the .pc.example files in config/. Test if pkg-config can find .pc files for all dependencies, including custom ones in config/:

$ make test

This version of Caffe is used, which works with SSD models. Thorough instructions are given there on how to install and configure Caffe. They refer to the official documentation on how to install prerequisites.

See Caffe above.

• data/ - Put datasets (images and/or detections) here. Images are expected to be in <path/to/sequence>/images/, and detections in <path/to/sequence>/model-type (this is also where custom detections will be put). Sequence maps should exist in data/seqmaps (an example file is given). Tracking output will be in data/result/<same/structure/as/dataset>/<sequence-name>.txt.
• models/ - To use Caffe, put models in models/. Config files for each model should exist in models/config/ (an example file is given).

Three example usages are provided:

1. Detect - Detect objects in a provided sequence of images. Objects are detected by a CNN (Caffe).
2. Track - Track objects from provided detections. The detections should be on the format used in MOTChallenge.
3. Detect and Track - Track objects detected in a provided sequence of images. Objects are detected by a CNN (Caffe).

All source code is in cpp/src/.

• To track objects from pre-existing detections, create an instance of MCSORT. No code in detector/ is needed. Use function tracker.track(<detections>).
• To detect and track objects from images, create an instance of ImageTracker. Requires Caffe. Use function tracker.detectAndTrack(<image>).