Accuracy for Siamese Network?

[Bradley Neuberg]

I'm training the Caffe Siamese network example (https://github.com/BVLC/caffe/tree/master/examples/siamese). This example has both training and validation data. I'd like to get a sense of the accuracy of the trained model on the validation data set. However, only the loss is reported.

Is it possible to add an AccuracyLayer to the Siamese network to also get a sense of how the trained model is doing using the validation data?

I tried the following:

layer {

  name: "accuracy"

  type: "Accuracy"

  bottom: "feat"

  bottom: "feat_p"

  bottom: "sim"

  top: "accuracy"

  include {

    phase: TEST

  }

}

However this doesn't work as only two bottom entries are allowed, not three ("feat" and "feat_p" roll up from the two halves of the trained network, while "sim" is the correct, expected target value).

Does the Siamese network have a predicted output of 0 or 1 that indicates similarity? I know there is the constrastive loss function that takes both halves of the Siamese network and attempts to determine how accurate the prediction is to the target:

layer {

  name: "loss"

  type: "ContrastiveLoss"

  bottom: "feat"

  bottom: "feat_p"

  bottom: "sim"

  top: "loss"

  contrastive_loss_param {

    margin: 1

  }

}

If I can roll this up to a single predicted 0 or 1 value I could then get an accuracy store vs. the target (I believe).

Any pointers on the right way to get accuracy values for these kinds of networks in Caffe? I'd like to get accuracy values every X number of steps on the validation data so I can graph it.

Best,

  Brad Neuberg

[Gavin Hackeling]

I am also interested in this.

[Youyou]

Hi Bradley and Gavin, have you guys figure that issue out?

[Bradley Neuberg]

It turns out that having an accuracy for a siamese network doesn't quite make sense. The siamese network learns a function that clusters similar images together and forces other images farther away. Once you have this function you then need to find a threshold value in terms of the distance between any two given images that returns whether they are the same person or object; you could find this threshold via an ROC curve.

I actually ended up abandoning siamese networks for my own application.

Best,

  Brad

--
You received this message because you are subscribed to a topic in the Google Groups "Caffe Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/caffe-users/D-7sRDw9v8c/unsubscribe.
To unsubscribe from this group and all its topics, send an email to caffe-users...@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/ee1b8982-2234-4168-a4f3-a710ddfeebdf%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[feelectronic...@gmail.com]

Hi,,, I know its very late to ask.

Is it possible that we compute the euclidean distance between the "feat" and "feat_p", making it into a 1xm dimension vector, and then pass this vector along with "sim" value to the Accuracy_layer? will the result make sense?

I checked the accuracy_layer.cpp file but I could not understand whats going inside that code. 

Thanks!

[levat...@gmail.com]

Hi,,I think your idea is good,i have the same trouble with accuracy of the siamese net.and now have you accomplished this?

在 2016年1月7日星期四 UTC+8下午1:49:23，feelectronic...@gmail.com写道：

[Shahnawaz Grewal]

Hi,

Did you guys able to get the accuracy for Siamese network?

@Bradley, can you  please guide how I could find this threshold via an ROC curve