Git Product home page Git Product logo

energy_ood's Introduction

Energy-based Out-of-distribution Detection (Energy OOD)

This repository is the official implementation of Energy-based Out-of-distribution Detection by Weitang Liu, Xiaoyun Wang, John Owens and Yixuan Li. This method is an effective and easy OOD detector with and without fine-tuning. Our code is modified from Outlier-Exposure.

image

Pretrained Models and Datasets

Pretrained models are provided in folder

./CIFAR/snapshots/

Please download the datasets in folder

./data/

Testing and Fine-tuning

run energy score testing for cifar10 WRN

bash run.sh energy 0

run energy score testing for cifar100 WRN

bash run.sh energy 1

run energy score training and testing for cifar10 WRN

bash run.sh energy_ft 0

run energy score training and testing for cifar100 WRN

bash run.sh energy_ft 1

Results

Our model achieves the following average performance on 6 OOD datasets:

1. MSP vs energy score with and without fine-tuned on CIFAR-10

Model name FPR95
Softmax score 51.04%
Energy score (ours) 33.01%
Softmax score with fine-tune 8.53%
Energy score with fine-tune (ours) 3.32%

2. CIFAR-10 (in-distribution) vs SVHN (out-of-distribution) Score Distributions

image

3. Performance among different baselines for WideResNet

CIFAR-10:

Model name FPR95
Softmax score 51.04%
Energy score (ours) 33.01%
ODIN 35.71%
Mahalanobis 37.08%
Outlier Exposure 8.53%
Energy score with fine-tune (ours) 3.32%

CIFAR-100:

Model name FPR95
Softmax score 80.41%
Energy score (ours) 73.60%
ODIN 74.64%
Mahalanobis 54.64%
Outlier Exposure 58.10%
Energy score with fine-tune (ours) 47.55%

Outlier Datasets

These experiments make use of numerous outlier datasets. Links for less common datasets are as follows, 80 Million Tiny Images Textures, Places365, LSUN-C, LSUN-R, iSUN and SVHN.

Citation

 @article{liu2020energy,
      title={Energy-based Out-of-distribution Detection},
      author={Liu, Weitang and Wang, Xiaoyun and Owens, John and Li, Yixuan},
      journal={Advances in Neural Information Processing Systems},
      year={2020}
 } 

energy_ood's People

Contributors

wetliu avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.