Git Product home page Git Product logo

lttm / il-semsegm Goto Github PK

View Code? Open in Web Editor NEW
58.0 6.0 2.0 5.04 MB

Code for the paper "Incremental Learning Techniques for Semantic Segmentation", Michieli U. and Zanuttigh P., ICCVW, 2019

Home Page: https://lttm.dei.unipd.it/paper_data/IL/

License: Other

Python 96.12% Batchfile 1.37% Shell 1.48% MATLAB 1.03%
incremental-learning semantic-segmentation catastrophic-forgetting knowledge-distillation

il-semsegm's Introduction

Incremental Learning Techniques for Semantic Segmentation

This is the code of the paper: Michieli U., Zanuttigh P., "Incremental Learning Techniques for Semantic Segmentation", Proceedings of the International Conference on Computer Vision (ICCV), Workshop on Transferring and Adapting Source Knowledge in Computer Vision (TASK-CV), Seoul (South Korea), 2 November 2019.

If you find the code or the paper useful for your research please cite our work as:

@inproceedings{michieli2019,
  title={Incremental Learning Techniques for Semantic Segmentation},
  author={Michieli, Umberto and Zanuttigh, Pietro},
  booktitle={International Conference on Computer Vision (ICCV), Workshop on Transferring and Adapting Source Knowledge in Computer Vision (TASK-CV)},
  year={2019}
}

The PDF of the paper can be found at: https://arxiv.org/abs/1907.13372

The webpage of the paper is: https://lttm.dei.unipd.it/paper_data/IL/

architecture image

Setup

  • Download the pretrained model of the Deeplab-v2 architecture pretrained on MSCOCO from: https://drive.google.com/drive/folders/0B_rootXHuswsZ0E4Mjh1ZU5xZVU and place it inside the folder pretrained/
  • Download the Pascal VOC dataset or your desired dataset and process it with the provided scripts for pre-processing (further info at sample_dataset/README.txt).
    • For a quick start simply download the tfrecords folder provided from the webpage of the paper (https://lttm.dei.unipd.it/paper_data/IL/) and place it in sample_dataset/
    • Alternatively, download the Pascal VOC 2012 and run sample_dataset/build_data.bat which will create 4 tfrecords files in sample_dataset/tfrecords/

Training

I suggest to start from the folder codes/LD1/ where you can run the simulations for the incremental scenario in which only the last class is added and only the LD1 loss is present.
The procedure is discussed in detail at codes/LD1/README.txt.
NB: for the other folders no README.txt exists since they are straightforward extensions of the code contained in the LD1/ folder

Software & Hardware

  • the code has been tested on Tensorlfow 1.10 - 1.14
  • the vast majority of the experiments have been conducted on a computer with:
    • processor: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz, 3701 Mhz, 6 Core(s), 12 Logical Processor(s)
    • RAM: 32 GB
    • OS: Microsoft Windows 10 Pro Education
    • GPU: NVIDIA GeForce RTX 2080 Ti, 11GB

Contact Information

I can provide checkpoints, datasets or other material upon specific request.

Umberto Michieli
Ph.D. Student
University of Padova
Department of Information Engineering (DEI)
Email: [email protected]
Personal Website: https://umbertomichieli.github.io/

License

This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

il-semsegm's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

il-semsegm's Issues

Bug in LD1

Hi! I have tested for LD1 recently. I found two errors and had corrected it.

  1. line 35 in script.job
    --validation_file ../../../sample_dataset/tfrecords/train_20.tfrecords
    should be
    --validation_file ../../sample_dataset/tfrecords/train_20.tfrecords

  2. UnboundLocalError: local variable 'G_final_loss_value' referenced before assignment in train_with_distillation.py
    There is the same bug as in issue#1. But I didn't find the answer to that problem.
    Here, I solved it by adding G_final_loss_value = 0 in line 171

FYI, thanks!

Problem of network_loss

Hi, I am trying to understand the function network_loss in model_with_distillation.py.
I'm confusing with the case applied_to == 'all', it seems to be the loss is applied to all classes, and the default num_class = 21.

My question is why do we have to apply the loss to the class belonging to all of the unseen classes?
For example, we first train the model for 1_19, and we have default num_class = 21. Thus, we are calculating the loss for 21 classes and we actually training on a model for 20 classes.

In short, the point is whether this configuration is identical to the equation (2) in Incremental Learning Techniques for Semantic Segmentation.

Looking forward to your reply. Thanks!

Display of visualized graphs in the paper

Hello, I would like to ask how the visualized graph in your paper is displayed?

First question. Can only show 5 new classes of segmentation graphs? Can the 15 classes of segmentation maps the model learned earlier be displayed?
Second question. Which code is the specific result of the 20 classes of segmentation in the article? Did you change 19 to 20 directly? How to save the segmentation map at this time?

Looking forward to your reply. Thank you.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.