Triplet loss with classification about deep-learning-recipes HOT 3 CLOSED

Hi @lkmklsmn -thanks for the questions!

The identity loss is only an engineering workaround for Keras to use the custom triplet loss: having this identity loss as a "decoy", we can just minimize the margin-based triplet loss (called in the lambda layer) or any custom loss we want to learn the embeddings. I'm not sure if there are more elegant solutions for this now, but this is probably the quickest one I can think of.

Conceptually, I'm not sure if such (Siamese) networks can be adapted to build a classifier. The prediction result will not be about which number category an image belongs to -- depending on the type of triplets you define, it's probably more about if two images represent the same number or not, and that is not strictly "classification".

For the prediction itself, there's probably no need to add extra layers. Remember our goal here is merely to learn the embeddings. After getting the embeddings, you'll be able to do predictions (on if two images are similar or not): https://github.com/road2stat/deep-learning-recipes/blob/master/triplet-loss-keras/metric-auc.R#L3-L13

I guess CNN would be a much better network architecture for image classification and learning image embeddings. The triplet loss is mostly used for special applications, such as the recommender system or face recognition problems. You can learn more about the face recognition problem and tell the difference between face recognition and image classification from Andrew Ng's lecture here: https://www.youtube.com/watch?v=d2XB5-tuCWU.

from deep-learning-recipes.

Thx for the reply!

I guess training a RF on the learned embedding would be one way to "add" classification to it, no?

However, any testing data will have to come in triplets, correct?
How would I construct those triplets? Take one test sample and compare with two classes (2 class classification) from the training data? Would this be overfitting?

from deep-learning-recipes.

I think that may depend on the specific problem and essentially, your definition of "classification". Take face recognition as an example, the face database has many classes (different people's registered faces), you trained the model with a triplet loss on this data. The testing input is a new face, and the output is: for each existing face, we predict the probability if the input face is the same face as the existing face (class). This can be considered as "classification" under the context of image recognition.

Only the training data is in the form of triplets. On testing data, for recommender systems, depending on what you have and you want to predict, the testing data can come in as a user, an item, or a user-item pair (or generally speaking, a user-item pair eventually). For face recognition, the testing data is usually a face image (as the "user", and the "item" would be all the existing faces in the database).

You can construct triplets offline or online, it can be as simple as random sampling, or as complicated as solving a standalone optimization problem. Here are some examples: https://omoindrot.github.io/triplet-loss

Again, if you only have two classes (or even many more classes) of images and their labels, then (I believe) the CNN would be a much more effective way to build a model for classification (also learning the layered representation for images), since it doesn't have the "hard negative mining" issue like the triplet loss models do.

from deep-learning-recipes.