-
ResNet (Deep Residual Learning for Image Recognition)
-
Pre-act ResNet (Identity mappings in deep residual networks)
-
ResNeXt (Aggregated Residual Transformations for Deep Neural Networks)
-
DenseNet (Densely Connected Convolutional Networks)
-
Train on CIFAR-10 and CIFAR-100 with ResNeXt29-8-64d and ResNeXt29-16-64d
-
Train on CIFAR-10 and CIFAR-100 with ResNet20,32,44,56,110
-
Train on CIFAR-10 and CIFAR-100 with Pre-Activation ResNet20,32,44,56,110
-
Train on CIFAR-10 and CIFAR-100 with DenseNet
-
Train ImageNet
To train on CIFAR-10 using 4 gpu:
python main.py ./data/cifar.python --dataset cifar10 --arch resnext29_8_64 --save_path ./snapshots/cifar10_resnext29_8_64_300 --epochs 300 --learning_rate 0.05 --schedule 150 225 --gammas 0.1 0.1 --batch_size 128 --workers 4 --ngpu 4Or there are some off-the-shelf scripts can dirrectly be used for training.
CUDA_VISIBLE_DEVICES=0,1,2,3 sh ./scripts/train_model.sh resnet20 cifar10Train the ResNet-18 on ImageNet with 8 GPUs
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 sh ./scripts/train_imagenet.sh resnet18A simplified CaffeNet-like model for CIFAR-10, which obtains the top1 accuracy of 89.5.
sh ./scripts/cifar10_caffe.sh
From the original ResNeXt and ResNet papers:
| depth | cardinality | base width | parameters | error cifar10 | error cifar100 | architecture |
|---|---|---|---|---|---|---|
| 29 | 8 | 64 | 34.4M | 3.65 | 17.77 | ResNeXt |
| 29 | 16 | 64 | 68.1M | 3.58 | 17.31 | ResNeXt |
| 20 | * | * | 0.27M | 8.75 | - | ResNet |
| 32 | * | * | 0.46M | 7.51 | - | ResNet |
| 44 | * | * | 0.66M | 7.17 | - | ResNet |
| 56 | * | * | 0.85M | 6.97 | - | ResNet |
| 110 | * | * | 1.7M | 6.61 | - | ResNet |
| 1202 | * | * | 19.4M | 7.93 | - | ResNet |
| depth | cardinality | base width | parameters | error cifar10 | error cifar100 | architecture |
|---|---|---|---|---|---|---|
| 29 | 8 | 64 | 34.4M | 3.67 | 17.66(17.47) | ResNeXt |
| 29 | 16 | 64 | 68.1M | 3.59(3.39) | 17.31(17.06) | ResNeXt |
| 20 | * | * | 0.27M | 8.47 | 32.99 | ResNet |
| 32 | * | * | 0.46M | 7.67 | 30.80 | ResNet |
| 44 | * | * | 0.66M | 7.23 | 29.45 | ResNet |
| 56 | * | * | 0.85M | 6.86 | 28.89 | ResNet |
| 110 | * | * | 1.7M | 6.62 | 27.62 | ResNet |
| 20 | * | * | 0.27M | 8.35 | 31.79 | Pre-Act |
| 32 | * | * | 0.46M | 7.57 | 30.02 | Pre-Act |
| 44 | * | * | 0.66M | 29.43 | Pre-Act |
| arch | Top-1 Accuracy | Top-5 Accuracy | Loss |
|---|---|---|---|
| ResNet-18 | 70.17 | 89.48 | 1.3097 |
| ResNet-18 | 70.22 | 89.43 | 1.5979 |
| ResNet-18 | 70.28 | 89.63 | 1.3023 |
| ResNet-34 | 73.92 | 91.62 | 1.0315 |
| ResNet-50 | 76.19 | 93.10 | 0.8172 |
- Torch (@facebookresearch). (Original) CIFAR and ImageNet
- MXNet (@dmlc). ImageNet
- PyTorch (@prlz77). CIFAR
- EraseReLU. (will be public soon)
@inproceedings{he2016deep,
title={Deep residual learning for image recognition},
author={He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian},
booktitle={Computer Vision and Pattern Recognition},
year={2016}
}
@inproceedings{he2016identity,
title={Identity mappings in deep residual networks},
author={He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian},
booktitle={European Conference on Computer Vision},
year={2016}
}
@inproceedings{xie2017aggregated,
title={Aggregated residual transformations for deep neural networks},
author={Xie, Saining and Girshick, Ross and Doll{\'a}r, Piotr and Tu, Zhuowen and He, Kaiming},
booktitle={Computer Vision and Pattern Recognition},
year={2017}
}
@inproceedings{huang2017densely,
title={Densely connected convolutional networks},
author={Huang, Gao and Liu, Zhuang and Weinberger, Kilian Q and van der Maaten, Laurens},
booktitle={Computer Vision and Pattern Recognition},
year={2017}
}
@article{dong2017eraserelu,
title={EraseReLU: A Simple Way to Ease the Training of Deep Convolution Neural Networks},
author={Dong, Xuanyi and Kang, Guoliang and Zhan, Kun and Yang, Yi},
journal={arXiv preprint arXiv:1709.07634},
year={2017}
}
The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) dataset has 1000 categories and 1.2 million images. The images do not need to be preprocessed or packaged in any database, but the validation images need to be moved into appropriate subfolders.
-
Download the images from http://image-net.org/download-images
-
Extract the training data:
mkdir train && mv ILSVRC2012_img_train.tar train/ && cd train
tar -xvf ILSVRC2012_img_train.tar && rm -f ILSVRC2012_img_train.tar
find . -name "*.tar" | while read NAME ; do mkdir -p "${NAME%.tar}"; tar -xvf "${NAME}" -C "${NAME%.tar}"; rm -f "${NAME}"; done
cd ..- Extract the validation data and move images to subfolders:
mkdir val && mv ILSVRC2012_img_val.tar val/ && cd val && tar -xvf ILSVRC2012_img_val.tar
wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent