Comments (6)
The BoT-SORT indeed is real-time.
It is the CMC that takes up most of the computation time.
For reference, you can check this.
When CMC is disabled, BoT-SORT is able to achieve 60+ FPS on Jetson NX in a sequence having an average of 226 objects/image.
from bot-sort.
I have the same issue.
But way more extreme in my case.
BoTSORT is not something I would use for real-time applications.
from bot-sort.
Ah strange, OC_SORT gets nearly 700 fps (no inference) on cpu and the implementation looks so similar.
I assume oc_sort doesn't compensate for camera motion? What's your findings for the most performant real time tracking algorithm @mikel-brostrom?
from bot-sort.
What's your findings for the most performant real time tracking algorithm @mikel-brostrom?
Both OCSORT and ByteTrack are absolutely the fastest, achieving around 800FPS on a mere Intel® Core™ i5-4210U CPU.
For best MOT metrics while real-time (20-30 FPS) it is still StrongSORT, at least based on my own MOT16 eval results:
Yolov5 StrongSORT
HOTA: StrongSORT HOTA DetA AssA DetRe DetPr AssRe AssPr LocA RHOTA HOTA(0) LocA(0) HOTALocA(0)
COMBINED 52.948 51.068 55.46 55.639 75.608 59.992 80.202 81.989 55.447 68.353 76.972 52.613
CLEAR: StrongSORT MOTA MOTP MODA CLR_Re CLR_Pr MTR PTR MLR sMOTA CLR_TP CLR_FN CLR_FP IDSW MT PT ML Frag
COMBINED 60.265 79.402 60.705 67.147 91.246 31.915 47.776 20.309 46.434 74135 36272 7112 486 165 247 105 2164
Identity: StrongSORT IDF1 IDR IDP IDTP IDFN IDFP
COMBINED 66.889 58.056 78.893 64098 46309 17149
FPS: 20
Yolov5 BoTSORT (no camera motion compensation, the implemented ones: ecc and orb, are too expensive computationally for real-time applications)
Used parameters for BoTSORT can be found here:
HOTA: BoTSORT HOTA DetA AssA DetRe DetPr AssRe AssPr LocA RHOTA HOTA(0) LocA(0) HOTALocA(0)
COMBINED 52.943 51.653 54.766 56.387 75.51 60.55 76.832 82.008 55.485 68.441 77.037 52.725
CLEAR: BoTSORT MOTA MOTP MODA CLR_Re CLR_Pr MTR PTR MLR sMOTA CLR_TP CLR_FN CLR_FP IDSW MT PT ML Frag
COMBINED 60.993 79.48 61.52 68.097 91.192 34.816 47.195 17.988 47.019 75184 35223 7262 582 180 244 93 2111
Identity: BoTSORT IDF1 IDR IDP IDTP IDFN IDFP
COMBINED 66.54 58.114 77.823 64162 46245 18284
FPS: 3
from bot-sort.
Hi @C-monC and @mikel-brostrom, sorry for the late response I wasn't available for the last month.
-
BoT-SORT can achieve real-time. I created a c++ real-life implementation that I, unfortunately, can't publish here.
The ORB and ECC method are indeed slower and thus I used the c++ OpenCV VideoStab GMC method, as you can see in the paper.
Currently, there is no python implementation for this module, this is why I write the GMC result to files and read it in runtime. For demo arbitrary videos (not from the MOTChallenge) I enabled the ECC and the ORB option. -
I just added another CMC method which is following the OpenCV GMC code and it is based on sparse Optical Flow.
This CMC method is running at ~16 mSec for a 1920x1080 frame on my computer. -
As I mentioned in the paper, the CMC module should run in parallel to the detector, thus the extra 16 mSec could be reduced to nearly zero latency (because the detector usually takes most of the runtime).
Therefore, the overall runtime should be governed by the detector time (YOLOv5 in your case) and the ReID module if it is enabled.
I hope I will have time soon to create a multi-threaded implementation in python which run the detector and the CMC in parallel. -
By the way, BoT-SORT (with correct CMC in parallel and without ReID) and ByteTrack should take the same runtime.
And also, BoT-SORT-ReID and StrongSORT should have the same runtime, especially because StrongSORT used ECC and ReID (if I remember correctly).
In any case, thank you for your interest in this work and I hope I was able to help you.
from bot-sort.
Hi @NirAharon.
Could you provide the real-time version of BoT-SORT?
Thank you!
from bot-sort.
Related Issues (20)
- CL.EXE -Application Error
- Prediction from Kalman
- How to train on custom
- How to train on custom dataset and evaluate the MOTA?
- How should I adjust parameters when tracking bbox results for low scores?
- Train with distributed mode_ multiple GPUs HOT 1
- Error: all query identities do not appear in gallery HOT 2
- reproducing demo but blank output
- No inf checks were recorded for this optimizer HOT 2
- Where is the option of Tracklet Interpolation ? HOT 1
- cv2.error: OpenCV(4.8.0) /io/opencv/modules/core/src/batch_distance.cpp:274: error: (-215:Assertion failed) type == src2.type() && src1.cols == src2.cols && (type == CV_32F || type == CV_8U) in function 'batchDistance'
- Warning with gmc.py, line 273, not enough matching point. HOT 1
- Has anyone tried to use DT-DETR or similar instead of YOLO in BoT-SORT? HOT 1
- About model training
- can yolov8 support?
- multi-camera-multi-object-detection with re-identification HOT 3
- How to train without Pre-trained Backbones like resnest50?
- Bug Report: duplicate STracks in the list tracker.removed_stracks
- MOT17 test results
- Small Object tracking
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bot-sort.