kdercksen / hunter2_dbi Goto Github PK

Maybe we could also use Googles image search API on our hard to predict doggos. This requires us to find the classes that are hard to predict and use the label as a query. Another option is to use a representative image for reverse image search. The last option could be quite challenging but this code snippet could be a helpful start:

import requests

filePath = '/data/train/doggo.png'
searchUrl = 'http://www.google.hr/searchbyimage/upload'
multipart = {'encoded_image': (filePath, open(filePath, 'rb')), 'image_content': ''}
response = requests.post(searchUrl, files=multipart, allow_redirects=False)
fetchUrl = response.headers['Location']

To get the images we can use the BeautifulSoup library to process the response of Google's API.

Keras has an option to supply a list of class weights to the fitting procedure; this allows 'learning more' from less represented classes. Seeing as our least common class has approximately half the examples of the most common class, we could maybe implement this to try and combat imbalance.

http://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html

(see also Keras Model.fit_generator documentation at https://keras.io/models/model/)

Just another idea... We could use multiple pretrained networks instead of just Inception v3, and concatenate their bottleneck layers using keras.layers.concatenate. This way we could combine features from multiple models to (hopefully? maybe??) increase accuracy. Basic idea:

from keras.layers import concatenate
inceptionv3 = inception_v3.InceptionV3(weights='imagenet', include_top=False)
resnet50 = resnet50.ResNet50(weights='imagenet', include_top=False)
base_output = concatenate([inceptionv3.output, resnet50.output])
# Add layers on top and train, maybe finetune top blocks etc

Maybe add this to improve performance...

x_train = (x_train - x_train.mean(axis=0)) / np.clip(x_train.std(axis=0), 1e-06, None)

https://docs.google.com/presentation/d/1BeV9PSan8i7IzZqH6w0Y1h__6fD7LadmcuwiunFafDg/edit?usp=sharing

Right now we freeze the Inception model as is, and only train two fully-connected layers on top to finetune to the new set of classes. We could also finetune some layers or blocks of the Inception model in addition to just stacking on new layers.

We can think about which layers or blocks to finetune and why, and implement additional finetuning in train.py. See also https://www.depends-on-the-definition.com/transfer-learning-for-dog-breed-identification/

Based on #10 ; if combining multiple networks is not viable or does not really improve performance, we can also ensemble other classifiers (such as random forests, clustering approaches etc). We should make a modular architecture in order to facilitate plugging in additional/new classifiers. Discussion below!

Helper functions to create submission file, load data, etc.

Think of ways to augment the original training data in order to improve performance. This goes back to #3 as well; augmentation on breeds that look alike can help the model distinguish between them more easily (hopefully).

This is a good place to start; additionally, we can implement custom augmentation (for example only for certain breeds or whatever).

Additional discussion can go in the comments!

Write a baseline model that we can use as a first submission to Kaggle. Idea:

• Use pre-trained neural net (https://www.kaggle.com/gaborfodor/dog-breed-pretrained-keras-models-lb-0-3)
• Use some classifier to classify features obtained from pre-trained model
• Create submission
• Submit

I recently had this idea... It may seem a bit hacky but hear me out :D

After training our model on the training set, we use it to make predictions about the test set. We can use these predictions as labels for the test set (although not 100% accurate), and use the labeled test set combined with the training data to further train the network. Assuming the classification error on the test set is already pretty good, this might provide the model with further clues about the nature of the data, and a second prediction might be even better. We can then use these improved labels to repeat the procedure, etc.

I don't know if this is worth looking into, I haven't done much research on it but intuitively it sounds like it may be promising (although a little cheaty, but not against the rules). Training the network in this way may take a bit longer than it already does, though... Anyone have any thoughts?

In order to improve on our simple initial approach, some deeper knowledge of the data is advantageous. We should try to find out about interesting properties of the data; we know that the dataset is fairly balanced, which is good, but there might be breeds that the model has trouble with. We can find out which breeds this is applicable to, and how to fix it.

Suggestions of techniques to use: pandas is great for exploring the data on a low or high level. Use a notebook (there is already one in the repository) to do some plotting, looking at the confusion matrix etc.

Additional discussion can go below in the comments!

RAM use is up to 23GB with the use of ImageDataGenerator... Should not be necessary.

See title

Please check off your name below if you've successfully installed everything and are able to train the network on your own machine.

• Koen Dercksen
• Sven den Hartog
• Denise Klep
• Jesse Zwamborn
• Alan R. Andrade
• Stephan Dooper

If you're having any difficulties with installation or whatever, post below! 😄