Real-time multi-object, segmentation and pose tracking using Yolov8 | Yolo-NAS with DeepOCSORT and LightMBN
This repo contains a collections of state-of-the-art multi-object trackers. Some of them are based on motion only, others on motion + appearance description. For the latter, state-of-the-art ReID model are downloaded automatically as well. Supported ones at the moment are: DeepOCSORT LightMBN, BoTSORT LightMBN, StrongSORT LightMBN, OCSORT and ByteTrack.
We provide examples on how to use this package together with popular object detection models. Right now only Yolov8 is available. Yolo-NAS & YOLOX coming soon.
Tutorials
Experiments
In inverse chronological order:
Everything is designed with simplicity and flexibility in mind. We don't hyperfocus on results on a single dataset, we prioritize real-world results. If you don't get good tracking results on your custom dataset with the out-of-the-box tracker configurations, use the examples/evolve.py script for tracker hyperparameter tuning.
pip install boxmot
Grab a coffee, this may take a few minutes
Click to expand!
Yolov8 model
$ python examples/track.py --yolo-model yolov8n.pt # bboxes only
yolov8n-seg.pt # bboxes + segmentation masks
yolov8n-pose.pt # bboxes + pose estimationTracking methods
$ python examples/track.py --tracking-method deepocsort
strongsort
ocsort
bytetrack
botsortTracking sources
Tracking can be run on most video formats
$ python examples/track.py --source 0 # webcam
img.jpg # image
vid.mp4 # video
path/ # directory
path/*.jpg # glob
'https://youtu.be/Zgi9g1ksQHc' # YouTube
'rtsp://example.com/media.mp4' # RTSP, RTMP, HTTP streamSelect Yolov8 model
There is a clear trade-off between model inference speed and overall performance. In order to make it possible to fulfill your inference speed/accuracy needs you can select a Yolov5 family model for automatic download. These model can be further optimized for you needs by the export.py script
$ python examples/track.py --source 0 --yolo-model yolov8n.pt --img 640
yolov8s.tflite
yolov8m.pt
yolov8l.onnx
yolov8x.pt --img 1280
...Select ReID model
Some tracking methods combine appearance description and motion in the process of tracking. For those which use appearance, you can choose a ReID model based on your needs from this ReID model zoo. These model can be further optimized for you needs by the reid_export.py script
$ python examples/track.py --source 0 --reid-model lmbn_n_cuhk03_d.pt
osnet_x0_25_market1501.pt
mobilenetv2_x1_4_msmt17.engine
resnet50_msmt17.onnx
osnet_x1_0_msmt17.pt
...Filter tracked classes
By default the tracker tracks all MS COCO classes.
If you want to track a subset of the classes that you model predicts, add their corresponding index after the classes flag,
python examples/track.py --source 0 --yolo-model yolov8s.pt --classes 16 17 # COCO yolov8 model. Track cats and dogs, onlyHere is a list of all the possible objects that a Yolov8 model trained on MS COCO can detect. Notice that the indexing for the classes in this repo starts at zero
MOT compliant results
Can be saved to your experiment folder runs/track/<yolo_model>_<deep_sort_model>/ by
python examples/track.py --source ... --save-txtTracker hyperparameter tuning
We use a fast and elitist multiobjective genetic algorithm for tracker hyperparameter tuning. By default the objectives are: HOTA, MOTA, IDF1. Run it by
$ python examples/evolve.py --tracking-method strongsort --benchmark MOT17 --n-trials 100 # tune strongsort for MOT17
--tracking-method ocsort --benchmark <your-custom-dataset> --objective HOTA # tune ocsort for maximizing HOTA on your custom tracking datasetThe set of hyperparameters leading to the best HOTA result are written to the tracker's config file.
Click to expand!
$ python examples/yolo_nas_track.py --source 0Click to exapand!
from boxmot import DeepOCSORT
tracker = DeepOCSORT()
cap = cv.VideoCapture(0)
while True:
ret, im = cap.read()
...
# dets:
# - your model's nms:ed outputs of shape Nx6 (x, y, x, y, conf, cls)
# im:
# - the original image (for better ReID results)
# - the downscaled one fed to you model (faster)
tracker_outputs = tracker.update(dets.cpu(), im) # --> (x, y, x, y, id, conf, cls)
...For Yolov8 tracking bugs and feature requests please visit GitHub Issues. For business inquiries or professional support requests please send an email to: [email protected]
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent