Loss for siamese network

[Swami]

Hi,

I am using a network having a siamese architecture that takes as input pairs of images and compute a loss value.

I have trained the network using contrastive loss. While testing, the loss is calculated in batch mode and the output loss for each batch

is an average per batch. For my purpose though I require the loss output for each instance of my test data. The way I am doing this is by modifying the

batch size to be 1 in the network definition so that the log file contains the loss values for each instance.

But I have around a million test instances and the this is very slow as opposed to using large batches. 

Is there a way to compute the loss value in large batches yet retrieve the loss for each individual instance ?

What would be the fastest way to do this ?

[Axel Angel]

By the lack of other idea, I would the patch the code to print out every loss in the for loop. Take a look at the code:

in ./src/caffe/layers/contrastive_loss_layer.cpp, in the function ContrastiveLossLayer<Dtype>::Forward_cpu:

  for (int i = 0; i < bottom[0]->num(); ++i) {

    dist_sq_.mutable_cpu_data()[i] = caffe_cpu_dot(channels,

        diff_.cpu_data() + (i*channels), diff_.cpu_data() + (i*channels));

    if (static_cast<int>(bottom[2]->cpu_data()[i])) {  // similar pairs
      // print this value v
      printf("1-loss: %f", dist_sq_.cpu_data()[i]);

      loss += dist_sq_.cpu_data()[i];

    } else {  // dissimilar pairs

      // print this value v
      double v = std::max(margin-dist_sq_.cpu_data()[i], Dtype(0.0));
      printf("1-loss: %f, v)

      loss += v;

    }

  }

This will print a LOT of values, you better write it to a file otherwise it could slow down the training. The code is not tested, adapt to your needs.

[Swami]

Thanks for your response. I guess I was not clear in my post. I want to compute loss per instance only during test time and not during training.

[Axel Angel]

I had the same issue and decided to just filter the log at the end. For my case I had different batch size for train versus test, thus I could print it and filter based on this number. I am not sure but maybe you can make a conditional "if" on the current mode (the include phase TRAIN versus TEST).