Build a Traffic Sign Recognition Project

The goals / steps of this project are the following:

• Load the data set (see below for links to the project data set)
• Explore, summarize and visualize the data set
• Design, train and test a model architecture
• Use the model to make predictions on new images
• Analyze the softmax probabilities of the new images
• Summarize the results with a written report

• The size of training set is 34799
• The size of the validation set is 4410
• The size of test set is 12630
• The shape of a traffic sign image is (32, 32, 3)
• The number of unique classes/labels in the data set is 43

Below are randomly chosen images from the training data set.

Distribution of training data among classes:

As a first step we performed data augmentation to balance the data. Two affine transformations were used: rotation by -15 or 15 degree and adding gaussian noise. As the result, the classes which had number of instances less then the mean were doubled or tripled depending on the initial amount. After augmenting, the distribution of training data is more balanced:

After augmenting the training dataset, images were converted to the grayscale to represent more contrast. The function cvtColor of the library opencv was used for converting:

cv2.cvtColor(x, cv2.COLOR_RGB2GRAY)

Example of a traffic sign image before and after grayscaling:

Augmented image	Augmented grayscaled image

At the next stage, we normalized the image data to make the data with mean zero and equal variance using the function normalize from the opencv library. The normalized images were reshaped form (32,32) to (32,32,1) using numpy:

np.expand_dims(cv2.normalize(x,  np.zeros((32, 32, 1)), 0, 255, cv2.NORM_MINMAX), -1)

The model was trained with a batch size of 128 for 100 epochs and learning rate of 0.001, AdamOptimizer and softmax cross entropy objective function.

The final model's performance results are:

• training set accuracy of 0.989
• validation set accuracy of 0.968
• test set accuracy of 0.952

The architecture of the NN we used to address classification of the traffic signs is Lenet-5 with rgb normalized images. Based on the high training accuracy (99%) and lower validation accuracy (90%), we concluded that the model suffered from overfitting. To avoid overfitting we added to the initial architecture two dropouts layers with a keeping probability of 0.5. Dropout is added after the first pooling layer and befor the last fully-connected layer. Augmenting the training dataset, adding dropouts and tuning the number of epochs resulted in achieving 96.8% validation accuracy. Considering the semantic complexity of the images (the images are quite simple) and their small size (32x32), two convolutional layers performed well.

The images were chosen to test the network against different lighting conditions for traffic signs. The first and third images were chosen as examples of a dark image. The second, fourth and fifth images have similar lighting conditions.

Prediction results:

The model was able to correctly guess 5 of the 5 traffic signs, which gives an accuracy of 100%. This compares favorably to the accuracy on the test set of 95%.