CORe50

A new Dataset and Benchmark for Continuous Object Recognition

WARNING: This repository is still under construction!

• CORe50 core code-base
• CORe50 benchmark configuration files
• Easy-to-access results data and baches configurations
• Easy-setup and getting started
• Reproducibility tests (in progress...)

In this page we provide the code and all the materials related to the CORe50 benchmark. If you plan to use this dataset or other resources you'll find in this page, please cite our latest paper:

@article{lomonaco2017core50,
   title={CORe50: a New Dataset and Benchmark for Continuous Object Recognition},
   author={Lomonaco, Vincenzo and Maltoni, Davide},
   journal={arXiv preprint arXiv:1705.03550},
   year={2017}
}

You can find more information about the dataset and benchmark at as well as additional data to download at: vlomonaco.github.io/core50.

Dependencies

In order to extecute the code in the repository you'll need to install the following dependencies in a Python 2.7 environment:

• Numpy: Matrices operations and stuff

pip install numpy

• Sacred: Experiments Manager

pip install sacred

• Caffe: Actual DL back-end (easily interchangeable)

Follow the step-by-step guide for installing caffe here.

Project Structure

Up to now the projects is structured as follows:

• confs/: In this folder you can find all the experiments configurations and the caffe definition files. sI, sII and sIII stand for the NI, NC and NIC scenarios, respectively.
• core/: The actual code of the benchmark.
• data/: Void at first. After the setup it will be filled with data needed for the experiments. It will also be used for storing partial computations.
• extras/: Results and configuration files you can download without delving into the code.
• scripts/: Currently void. It will contain useful scripts to help you downloading the necessary extra materials. Bash scripts will be updated soon for running the benchmark without much pain.
• LICENSE: Standard Creative Commons Attribution 4.0 International License.
• README.md: This instructions file.
• run_sI_exps.sh: Simple bash script for running the "New Instances (NI)" experiments with the different architectures and strategies
• run_sII_exps.sh: Simple bash script for running the "New Classes (NC)" experiments with the different architectures and strategies
• run_sIII_exps.sh: Simple bash script for running the "New Instances and Classes (NIC)" experiments with the different architectures and strategies

Getting Started

In order to run the experiments and reproduce the benchmark first of all we need to download the pre-trained models and the CORe50 dataset. This can be automatically done using the script provided:

./scripts/fetch_data.sh

All the data will be downloaded in the data/ directory. After this initial step you can directly run the experiments with the bash scripts run_sI_exps.sh, run_sII_exps.sh and run_sIII_exps.sh for the NI, NC and NIC scenarios respectively.

For example, reproducing the first scenario experiments can be as easy as running:

./run_sI_exps.sh

Since this experiments can take a while (also more than 24h depending on the scenario) you can also disable some experiments just by commenting them in the bash script.

Troubleshooting

• If you find different results from out benchmark (for a few percentage points) that is to be expected! First of all because we use the cudnn engine which is not fully deterministic for convolutions. Second because the error may be accumulated during the incremental learning process. If you want full reproducibility (which means a ~2x in terms of time needed) just set the engine param of convolution to 1.

• If you find some trouble with the freezeweights strategy this is probably because you need to reset the learning rate multipliers in the prototxt (sorry, my bad.. I'm currently working on a new version of the code for creating the prototxt files instead of modifying them on the fly.).

• Hei! If you find any trouble don't get frustrated, just ask, we'll answer in a few hours! :-)

License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Author

• Vincenzo Lomonaco - email: [email protected]