Ground truth modification

[Alex Ter-Sarkisov]

I wrote a module for Caffe that extracts some data from the final score layer in FCN and uses it to modify ground truth mask. The modified GT mask is then used the normal way - to compute log softmax loss and backprop the gradients. Does this violate any rules of DL (i.e. using network information to change gt)? 

 

[Thomio Watanabe]

In theory the GT should be perfect.
If you were working with clasification, a cat is a cat and a dog is a dog (doesnt make sense to change the labels).
In segmentation there are some errors to delineate the objects.
If you are able to improve GT contours somehow you did something really good.
But I dont see how a CNN can improve something that it is still learning to do.
Also note FCN does not provide accurate results.

[Alex Ter-Sarkisov]

Yeah that's what I did: a module computes the overlap between argmax binary mask from a score map and gt mask modifies the gt mask in a specific way. The modified gt mask is used instead of the 'old' gt. I hope the explanation wasn't too bad)  

[Przemek D]

So the model (which in the beginning is pretty dumb, we can assume, as it's still learning the task) makes some prediction (which can as well be random), then uses that prediction to modify the actual true answer, and then uses that as a feedback. I don't really understand how exactly is the GT mask transformed, but unless you're doing something really clever, this doesn't seem to make sense. After all, imagine a student who's trying to learn some material from a book (GT), and during a rehearsal modifies the actual content of the book basing on their own imperfect answers - and then attempts to learn from the book they modified. IMHO, either this has no chance of working, or we're not understanding it correctly.