Achieved > 94% of accurracy for CIFAR10 dataset with only 50 epochs.

By Danh Doan

This repository serves my purpose of implementing and experiencing different modern Convolutional Neural Networks and using them to solve the well-known CIFAR10 dataset. MXNet is used as the main framework for Deep Learning.

When conducting experiment with CNN architectures, I use the same training parameters to draw a comparison between various CNNs. To efficiently utilize each network, experiment with another training parameters.

All networks are trained end-to-end and are implemented from scratch. Besides, Batch Normalization and Drop Out layers are applied whenever possible to increase the Accuracy and avoid Overfitting.

1-Cycle schedule is utilized in the training procedure. The value of 1-Cycle's parameters are analyzed after performing LR range test. In my schedule, the Triangle cycle part governs 40 epochs and the Cool-Down follows in the last 10 epochs. Depicting in the following figure:

• AlexNet:

• VGG13:

• ResNet34:

• DenseNet161:

• Inception-V3:

• 2019, Aug 20:

  • Apply 1-Cycle for Learning Rate Scheduler [paper]
  • Re-train all models with only 50 epochs and still achieve comparable accurracy or over higher

• 2019, Aug 16:

  • Apply LR scheduler built-in module from MXNet

• 2019, Aug 13:

  • Implement and Test with all Inception V3 architectures [paper]

• 2019, Aug 12:

  • Implement and Test with all GoogleNet architectures [paper]

• 2019, Aug 10:

  • Implement and Test with all DenseNet architectures [paper]

• 2019, Aug 8:

  • Implement and Test with all ResNet architectures [paper]
  • Implement and Test with all AlexNet architectures [paper]

• 2019, Aug 7:

  • Set up the training and test program
  • Implement and Test with all VGG architectures [paper]

• Install MXNet framework and GluonCV toolkit

  • For CPU only:

    pip install mxnet gluoncv

  • For GPUs

    pip install mxnet-cu90 gluoncv

    Change to match with CUDA version. mxnet-cu100 if CUDA 10.0 is installed