UPDATE:
[2022.03.03] Update the toolkit installation, dataset download instructions and a concise example. Now the basic function of this toolkit has been finished.

This repository contains the official python toolkit for running experiments and evaluate performance on VideoCube benchmark. The code is written in pure python and is compile-free.

VideoCube is a high-quality and large-scale benchmark to create a challenging real-world experimental environment for Global Instance Tracking (GIT) task. If you use the VideoCube database or toolkits for a research publication, please consider citing:

@ARTICLE{9720246,
author={Hu, Shiyu and Zhao, Xin and Huang, Lianghua and Huang, Kaiqi},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
title={Global Instance Tracking: Locating Target More Like Humans}, 
year={2022},
volume={},
number={},
pages={1-1},
doi={10.1109/TPAMI.2022.3153312}}

 [Project][PDF]

• VideoCube Python Toolkit
  • Table of Contents
    • Toolkit Installation
    • Dataset Download
    • A Concise Example
      • How to Define a Tracker?
      • How to Run Experiments on VideoCube?
      • How to Evaluate Performance?
    • Issues
    • Contributors

Clone the repository and install dependencies:

git clone https://github.com/huuuuusy/videocube-toolkit.git
pip install -r requirements.txt

Then directly copy the videocube folder to your workspace to use it.

Please view the Download page in the project website.

The VideoCube dataset includes 500 sequences, divided into three subsets (train/val/test). The content distribution in each subset still follows the 6D principle proposed in the GIT paper.

The dataset download and file organization process is as follows：

• Download three subsets (train/val/test) and the info data via Download page in the project website.

• Check the number of files in each subset and run the unzipping script. Before unzipping:

  • the train subset should includ 456 files (455 data files and an unzip_train bash)

  • the val subset should includ 69 files (68 data files and an unzip_val bash)

  • the test subset should includ 140 files (139 data files and an unzip_test bash)

• Run the unzipping script in each subset folder, and delete the script after decompression.

• Taking val subset as an example, the folder structure should follow:

|-- val/
|  |-- 005/
|  |  |-- frame_005/
|  |  |  |-- 000000.jpg/
|  |  |      ......
|  |  |  |-- 016891.jpg/
|  |-- 008/
|  |   ......
|  |-- 486/
|  |-- 493/

• Unzip attribute.zip in info data. Attention that we only provide properties files for train and val subsets. For ground-truth files, we only offer a small number of annotations (restart frames) for sequences that belong to the test subset. Please upload the final results to the server for evaluation.

• Rename and organize folders as follows：

|-- VideoCube/
|  |-- data/
|  |  |-- train/
|  |  |  |-- 002/
|  |  |  |   ......
|  |  |  |-- 499/
|  |  |-- val/
|  |  |  |-- 005/
|  |  |  |   ......
|  |  |  |-- 493/
|  |  |-- test/
|  |  |  |-- 001/
|  |  |  |   ......
|  |  |  |-- 500/
|  |  |-- train_list.txt
|  |  |-- val_list.txt
|  |  |-- test_list.txt
|  |-- attribute/
|  |  |-- absent/
|  |  |-- color_constancy_tran/
|  |  |   ......
|  |  |-- shotcut/

test.py is a simple example on how to use the toolkit to define a tracker, run experiments on VideoCube and evaluate performance.

To define a tracker using the toolkit, simply inherit and override init and update methods from the Tracker class. You can find an example in this page. Here is a simple example:

from videocube.trackers import Tracker

class IdentityTracker(Tracker):
    def __init__(self):
        super(IdentityTracker, self).__init__(
            name='IdentityTracker',  # tracker name
        )
    
    def init(self, image, box):
        self.box = box

    def update(self, image):
        return self.box

Instantiate an ExperimentVideoCube object, and leave all experiment pipelines to its run method:

from videocube.experiments import ExperimentVideoCube

# ... tracker definition ...

# instantiate a tracker
tracker = IdentityTracker()

# setup experiment (validation subset)
experiment = ExperimentVideoCube(
  root_dir='SOT/VideoCube', # VideoCube's root directory
  save_dir= os.path.join(root_dir,'result'), # the path to save the experiment result
  subset='val', # 'train' | 'val' | 'test'
  repetition=1 
)
experiment.run(
  tracker, 
  visualize=False,
  save_img=False, 
  method='restart' # method in run function means the evaluation mechanism, you can select the original mode (set none) or the restart mode (set restart)
  )

For evaluation in OPE mechanism, please use the report method of ExperimentVideoCube for this purpose:

# ... run experiments on VideoCube ...

# report tracking performance
experiment.report([tracker.name],attribute_name)

For evaluation in R-OPE mechanism, please use the report and report_robust method of ExperimentVideoCube for this purpose:

# ... run experiments on VideoCube ...

# report tracking performance
experiment.report([tracker.name],attribute_name)
experiment.report_robust([tracker.name])

Attention, when evaluated on the test subset, you will have to submit your results to the evaluation server for evaluation. The report function will generate a .zip file which can be directly uploaded for submission. For more instructions, see submission instruction.

See public evaluation results on VideoCube's leaderboard (OPE Mechanism) and VideoCube's leaderboard (R-OPE Mechanism).

Please report any problems or suggessions in the Issues page.

• Shiyu Hu
• Lianghua Huang