Build a Traffic Sign Recognition Project

The goals / steps of this project are the following:

• Load the data set (see below for links to the project data set)
• Explore, summarize and visualize the data set
• Design, train and test a model architecture
• Use the model to make predictions on new images
• Analyze the softmax probabilities of the new images
• Summarize the results with a written report

Here is a link to my project code

The dataset used was the german traffic signs dataset

• The size of training set: 34799 images
• The size of the validation set is: 4410 images
• The size of test set is : 12630
• The shape of a traffic sign image is : (32, 32, 3)
• The number of unique classes/labels in the data set is : 43

Here is an exploratory visualization of the data set. It is a histogram showing the distribution of classes within the training dataset.

I considered different types of normalization, data preprocessing, but ended up doing very minimal preprocessing after empirically testing it. Things I considered were data normalization, data augmentation through scaling/translation/salt-and-pepper-noise. However, these ended up hurting the validation set. Here is the code/results for some of the pre-processing that I tested but ended up not using.

![alt text][normalization-code]

I tried various forms of normalization such as (pixel - 128) / 180 and pixel / 255, but decided not to use any because I got the same results with batch norm.

My final model consisted of the following layers:

To train the model, I trained it for 40 epochs with an AdamOptimizer, a learning rate of 0.001. My loss was the cross entropy loss along with an L2 loss that I enforced on the weights.

My final model results were:

• training set accuracy of: 0.99
• validation set accuracy of: 0.950
• test set accuracy of: 0.946

If an iterative approach was chosen:

• What was the first architecture that was tried and why was it chosen? The first architecture that I chose was LeNet. Without any modification, it received a validation accuracy of 0.89.

• What were some problems with the initial architecture? I wrote code to figure out if it was overfitting/underfitting. Turns out, the training set accuracy of the initial model was high (0.97), but the validation set accuracy was only 0.87, thus I figured we needed to generalize the model.

• How was the architecture adjusted and why was it adjusted? To generalize, first I tried data augmentation. This meant normalizing the data, translating/horizontal/noise, however this did very little to increase the validation accuracy and actually hurt both accuracy. Then, I added a L2 loss to the weights of the FC and Conv Layers, and this improved it a little bit. Along, the way, I also played along with the Resnet architecture from this repo, however the problem lies in overfitting not underfitting so the higher capacity resnet didn't do much. Finally, I added batchnorm layers and this was the thing which helped fix the genralization error.

• Which parameters were tuned? How were they adjusted and why? I read a recent paper about how Batch Norm works works, and their novel thesis was that BatchNorm helps train networks faster/generalizes better because it allows models to use higher learning rates. Thus, I experimented with many different learning rates [0.001, 0.005, 0.01], and indeed with higher learning rates, the model was still able to learn. Moreover, because the loss contained both cross-entropy and L2_loss, I tuned how much L2_loss.

• What are some of the important design choices and why were they chosen? For example, why might a convolution layer work well with this problem? How might a dropout layer help with creating a successful model? A convolution layer works well on this problem because they allow for parameter sharing, extra local features(spatial), and are equivalent to small translations(with maxpooling)...

Here are five German traffic signs that I found on the web:

For some of the images like 30km/hr signs, do look like 80km/hr, which is another sign in the dataset.

Here are the results of the prediction:

The model was able to correctly guess 3 of the 5 traffic signs, which gives an accuracy of 60%. My test set accuracy is 95%, but within only 5 images, its hard to tell. One of the harder ones to do was the children crossing. The image seems to be rotated, and convolutions are not very rotationally invariant.

The code for making predictions on my final model in the part 5 of the jupyter notebook. For the children crossing image, the image was rescaled to be very small -- the neural network had a very small confidence in predicting the image

For the Keep Right image, the neural network predicted "Straight/Right", which is relatively close to the "Keep Right" image.