This is the implementation of a series of basic algorithms which is useful for video understanding, including Single Object Tracking (SOT), Video Object Segmentation (VOS), etc.
Current implementation list:
Example SiamFC++ outputs.
Example SAT outputs.
Please refer to SOT_SETUP.md
# demo with web camera
python3 ./demo/main/video/sot_video.py --config 'experiments/siamfcpp/test/vot/siamfcpp_alexnet.yaml' --device cuda --video "webcam"
# demo with video file, and dump result into video file (optional)
python3 ./demo/main/video/sot_video.py --config 'experiments/siamfcpp/test/vot/siamfcpp_alexnet.yaml' --device cuda --video $video_dir/demo.mp4 --output $dump_path/result.mp4
# demo with extracted image files, and dump result into image files (optional)
python3 ./demo/main/video/sot_video.py --config 'experiments/siamfcpp/test/vot/siamfcpp_alexnet.yaml' --device cuda --video $video_dir/*.jpg --output $dump_dirPlease refer to SOT_TEST.md for detail.
Please refer to SOT_TRAINING.md for detail.
project_root/
├── experiments # experiment configurations, in yaml format
├── main
│ ├── train.py # trainng entry point
│ └── test.py # test entry point
├── video_analyst
│ ├── data # modules related to data
│ │ ├── dataset # data fetcher of each individual dataset
│ │ ├── sampler # data sampler, including inner-dataset and intra-dataset sampling procedure
│ │ ├── dataloader.py # data loading procedure
│ │ └── transformer # data augmentation
│ ├── engine # procedure controller, including traiing control / hp&model loading
│ │ ├── monitor # monitor for tasks during training, including visualization / logging / benchmarking
│ │ ├── trainer.py # train a epoch
│ │ ├── tester.py # test a model on a benchmark
│ ├── model # model builder
│ │ ├── backbone # backbone network builder
│ │ ├── common_opr # shared operator (e.g. cross-correlation)
│ │ ├── task_model # holistic model builder
│ │ ├── task_head # head network builder
│ │ └── loss # loss builder
│ ├── pipeline # pipeline builder (tracking / vos)
│ │ ├── segmenter # segmenter builder for vos
│ │ ├── tracker # tracker builder for tracking
│ │ └── utils # pipeline utils
│ ├── config # configuration manager
│ ├── evaluation # benchmark
│ ├── optim # optimization-related module (learning rate, gradient clipping, etc.)
│ │ ├── optimizer # optimizer
│ │ ├── scheduler # learning rate scheduler
│ │ └── grad_modifier # gradient-related operation (parameter freezing)
│ └── utils # useful tools
└── README.md
For detail, please refer to markdown files under docs.
- SOT_SETUP.md: instructions for setting-up
- SOT_MODEL_ZOO.md: description of released sot models
- SOT_TRAINING.md: details related to training
- SOT_TEST.md: details related to test
- VOS_SETUP.md: instructions for setting-up
- VOS_MODEL_ZOO.md: description of released sot models
- VOS_TRAINING.md: details related to training
- VOS_TEST.md: details related to training
- DEVELOP.md: description of project design (registry, configuration tree, etc.)
- PIPELINE_API.md: description for pipeline API
- FORMATTING_INSTRUCTION: instruction for code formatting (yapf/isort/flake/etc.)
[] refine code stype and test cases
- video_analyst/evaluation/vot_benchmark and other related code have been borrowed from PySOT
- video_analyst/evaluation/got_benchmark and other related code have been borrowed from got-toolkit
- detectron2
Maintainer (sorted by family name):
- Xi Chen@XavierCHEN34
- Zuoxin Li@lzx1413
- Zeyu Wang@JWarlock
- Yinda Xu@MARMOTatZJU
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent