training problem about tripletloss HOT 5 OPEN

I also met this problem and solved it by removing the last ip layer, which is the "fc9_1" layer. Since the "fc9_1" layer will magnify the norm of the feature to enlarge the distance between a and an, you can see the distance keeps increasing untill it overflows. I also recommend adding a "PReLU" layer after the last ip layer before normalizing it.

from tripletloss.

@JoeFannie thanks .
Do you mean to remove the embedding layer?

layer {
name: "norm2"
type: "Python"
bottom: "fc7"
top: "norm2"
python_param {
module: "norm2layer"
layer: "Norm2Layer"
}
}
layer {
name: "fc9_1"
type: "InnerProduct"
bottom: "norm2"
top: "fc9_1"
param {
lr_mult: 1
decay_mult: 0
}
inner_product_param {
num_output: 512
weight_filler {
type: "xavier"
}
}
}

Then, what kind of layer do you measure?
( I need a dimensionally reduced vector. So I want to know how to reduce it.)

from tripletloss.

layer {
name: "norm2"
type: "Python"
bottom: "fc7"
top: "norm2"
python_param {
module: "norm2layer"
layer: "Norm2Layer"
}
}
add only norm2 layer is just good. If u want to reduce the dimension, u can move the norm2 layer to the next to the last ip layer "fc9_1". In one word, the feature is supposed to be normalized before being fed to the triplet loss layer. Hope it helps.

from tripletloss.

@JoeFannie sorry very late..and thanks.

fc7 -> fc9_1 -> norm2 ->"triplet_select"
Do you mean this order?

so, below...

,,,

layer {
  name: "fc7"
  type: "InnerProduct"
  bottom: "fc6"
  top: "fc7"
  param {
    lr_mult: 0
    decay_mult: 0
  }
  param {
    lr_mult: 0
    decay_mult: 0
  }
  inner_product_param {
    num_output: 4096
    weight_filler {
      type:"xavier"
    }
  }
}
layer {
  name: "relu7"
  type: "ReLU"
  bottom: "fc7"
  top: "fc7"
}
layer {
  name: "drop7"
  type: "Dropout"
  bottom: "fc7"
  top: "fc7"
  dropout_param {
    dropout_ratio: 0.5
  }
}

layer {
  name: "fc9_1"
  type: "InnerProduct"
  bottom: "fc7"
  top: "fc9_1"
  param {
    lr_mult: 1
    decay_mult: 0
  }
  inner_product_param {
    num_output: 512
    weight_filler {
      type: "xavier"
    }
  }
}

layer {
  name: "norm2"
  type: "Python"
  bottom: "fc9_1"
  top: "norm2"
  python_param {
    module: "norm2layer"
    layer: "Norm2Layer"
  }
}
#add activation layer may increase the feature's expression
layer {
  name: "triplet_select"
  type: "Python"
  bottom: "norm2"
  bottom: "labels"
  top: "archor"
  top: "positive"
  top: "negative"
  python_param {
    module: "tripletselectlayer"
    layer: "TripletSelectLayer"
  }
}
layer {
  name: "tripletloss"
  type: "Python"
  bottom: "archor"
  bottom: "positive"
  bottom: "negative"
  top: "loss"
  python_param {
    module: "tripletlosslayer"
    layer: "TripletLayer"
    param_str: "'margin': 0.2"
  }
  loss_weight: 1
}

from tripletloss.

It is what I mean

from tripletloss.