Transfer Learning using Adversarially Robust ImageNet Models
This repository contains the code and models necessary to replicate the results of our paper:
Do Adversarially Robust ImageNet Models Transfer Better?
Hadi Salman*, Andrew Ilyas*, Logan Engstrom, Ashish Kapoor, Aleksander Madry
Paper: https://arxiv.org/abs/2007.08489
Blog post: https://www.microsoft.com/en-us/research/blog/adversarial-robustness-as-a-prior-for-better-transfer-learning/
@InProceedings{salman2020adversarially,
title={Do Adversarially Robust ImageNet Models Transfer Better?},
author={Hadi Salman and Andrew Ilyas and Logan Engstrom and Ashish Kapoor and Aleksander Madry},
year={2020},
booktitle={ArXiv preprint arXiv:2007.08489}
}Getting started
Our code relies on the MadryLab public robustness library, which will be automatically installed when you follow the instructions below.
-
Clone our repo:
git clone https://github.com/microsoft/robust-models-transfer.git -
Install dependencies:
conda create -n robust-transfer python=3.7 conda activate robust-transfer pip install -r requirements.txt
Running transfer learning experiments
The entry point of our code is main.py (see the file for a full description of arguments).
1- Download one of the pretrained robust ImageNet models, say an L2-robust ResNet-18 with ε = 3. For a full list of models, see the section below!
mkdir pretrained-models &
wget -O pretrained-models/resnet-18-l2-eps3.ckpt "https://huggingface.co/madrylab/robust-imagenet-models/resolve/main/resnet18_l2_eps3.ckpt"
2- Run the following script (best parameters for each dataset and architecture can be found here)
python src/main.py --arch resnet18 \
--dataset cifar10 \
--data /tmp \
--out-dir outdir \
--exp-name cifar10-transfer-demo \
--epochs 150 \
--lr 0.01 \
--step-lr 30 \
--batch-size 64 \
--weight-decay 5e-4 \
--adv-train 0 \
--model-path pretrained-models/resnet-18-l2-eps3.ckpt
--freeze-level -1
--freeze-level -> -1: full-network transfer | 4: fixed-feature transfer
3- That's it!
Datasets that we use (see our paper for citations)
- aircraft (Download)
- birds (Download)
- caltech101 (Download)
- caltech256 (Download)
- cifar10 (Automatically downloaded when you run the code)
- cifar100 (Automatically downloaded when you run the code)
- dtd (Download)
- flowers (Download)
- food (Download)
- pets (Download)
- stanford_cars (Download)
- SUN397 (Download)
To use any of these datasets in the code:
- Download (click or use
wget) and extract the desired dataset somewhere, e.g.tar -xvf pets.tar mkdir /tmp/datasets mv pets /tmp/datasets - Add the dataset name and path as arguments , e.g.
python src/main.py --arch resnet18 ... --dataset pets --data /tmp/datasets/pets
Architectures
You can choose an architecture to use by simply passing it as arguments to the code e.g.
python src/main.py --arch resnet50 ...
The set of possible architectures is:
archs = [resnet18,
resnet50,
wide_resnet50_2,
wide_resnet50_4,
densenet,
mnasnet,
mobilenet,
resnext50_32x4d,
shufflenet,
vgg16_bn
]Download our robust ImageNet models
If you find our pretrained models useful, please consider citing our work.
Standard Accuracy of L2-Robust ImageNet Models
| Model | ε=0 | ε=0.01 | ε=0.03 | ε=0.05 | ε=0.1 | ε=0.25 | ε=0.5 | ε=1.0 | ε=3.0 | ε=5.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| ResNet-18 | 69.79 | 69.90 | 69.24 | 69.15 | 68.77 | 67.43 | 65.49 | 62.32 | 53.12 | 45.59 |
| ResNet-50 | 75.80 | 75.68 | 75.76 | 75.59 | 74.78 | 74.14 | 73.16 | 70.43 | 62.83 | 56.13 |
| Wide-ResNet-50-2 | 76.97 | 77.25 | 77.26 | 77.17 | 76.74 | 76.21 | 75.11 | 73.41 | 66.90 | 60.94 |
| Wide-ResNet-50-4 | 77.91 | 78.02 | 77.87 | 77.77 | 77.64 | 77.10 | 76.52 | 75.51 | 69.67 | 65.20 |
| Model | ε=0 | ε=3 |
|---|---|---|
| DenseNet | 77.37 | 66.98 |
| MNASNET | 60.97 | 41.83 |
| MobileNet-v2 | 65.26 | 50.40 |
| ResNeXt50_32x4d | 77.38 | 66.25 |
| ShuffleNet | 64.25 | 43.32 |
| VGG16_bn | 73.66 | 57.19 |
Standard Accuracy of Linf-Robust ImageNet Models
| Model | ε=0.5/255 | ε=1/255 | ε=2/255 | ε=4/255 | ε=8/255 |
|---|---|---|---|---|---|
| ResNet-18 | 66.13 | 63.46 | 59.63 | 52.49 | 42.11 |
| ResNet-50 | 73.73 | 72.05 | 69.10 | 63.86 | 54.53 |
| Wide-ResNet-50-2 | 75.82 | 74.65 | 72.35 | 68.41 | 60.82 |
We are hosting these models on HuggingFace too, check them out!
Maintainers
Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.
![microsoft-github-operations[bot] avatar](https://avatars.githubusercontent.com/in/41902?v=4 "microsoft-github-operations[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent