changliu19 / refer Goto Github PK

This API is able to load all 4 referring expression datasets, i.e., RefClef, RefCOCO, RefCOCO+ and RefCOCOg. They are with different train/val/test split by UNC, Google and UC Berkeley respectively. We provide all kinds of splits here. Note, RefCOCO+ may change in the future as we are still cleaning it. Notification will be announced if we made changes.

If you used the following three datasets RefClef, RefCOCO and RefCOCO+ that were collected by UNC, please consider cite our EMNLP2014 paper; if you want to compare with our recent results, please check our ECCV2016 paper.

Kazemzadeh, Sahar, et al. "ReferItGame: Referring to Objects in Photographs of Natural Scenes." EMNLP 2014.
Yu, Licheng, et al. "Modeling Context in Referring Expressions." ECCV 2016.

To install this package, along its dependencies, you can execute:

pip install -U .

This package depends on Numpy, Matplotlib, scikit-image and Cython, it also depends on the mscoco API mask routines, which are compiled during setup. These mask-related codes are copied from mscoco API.

Download the cleaned data and extract them into "data" folder

• 1. http://bvisionweb1.cs.unc.edu/licheng/referit/data/refclef.zip
• 2. http://bvisionweb1.cs.unc.edu/licheng/referit/data/refcoco.zip
• 3. http://bvisionweb1.cs.unc.edu/licheng/referit/data/refcoco+.zip
• 4. http://bvisionweb1.cs.unc.edu/licheng/referit/data/refcocog.zip

Besides, add "mscoco" into the data/images folder, which can be from mscoco COCO's images are used for RefCOCO, RefCOCO+ and refCOCOg. For RefCLEF, please add saiapr_tc-12 into data/images folder. We extracted the related 19997 images to our cleaned RefCLEF dataset, which is a subset of the original imageCLEF. Download the subset and unzip it to data/images/saiapr_tc-12.

The refer module (referit/refer.py) is able to load all 4 datasets with different kinds of data split by UNC, Google and UC Berkeley.

from referit import REFER

# locate your own data_root, and choose the dataset_splitBy you want to use
refer = REFER(data_root, dataset='refclef',  split_by='unc')
refer = REFER(data_root, dataset='refclef',  split_by='berkeley')  # 2 training and 1 testing images missed
refer = REFER(data_root, dataset='refcoco',  split_by='unc')
refer = REFER(data_root, dataset='refcoco',  split_by='google')
refer = REFER(data_root, dataset='refcoco+', split_by='unc')
refer = REFER(data_root, dataset='refcocog', split_by='google')  # testing data haven't been released yet
refer = REFER(data_root, dataset='refcocog', split_by='umd') # train/val/test split provided by UMD (recommended)