Calculation an accuracy of a binary classification of a network with one output

[Andriy]

What layer can be used to calculate an accuracy of a binary classification of a network with one output? This accuracy layer don't calculate it correctly.

[Przemek D]

A net with one output is more like a regression, presumably with a 0-1 range. I understand that you then make a binary decision basing on the regression output, picking one of two options.

Unfortunately this is not how Accuracy layer works, or any of Caffe if we're talking about classification, actually. If you're classifying between N classes, the net has to have N outputs. The Accuracy layer basically checks whether index of the most active of those N outputs is the same as the label.

Could you please tell us how do you make the binary decision from the single output? It's rather difficult to give you any concrete hints without knowing this. I'm tempted to say "change your net to 2 outputs and a softmax" but I assume there is a strong reason to not do it this way.

[Andriy]

Could you please tell us how do you make the binary decision from the single output? It's rather difficult to give you any concrete hints without knowing this.

Typically, binary classification tasks are solved using a single-output network because samples of two classes are separated one hyperlane but not two. The decision rule is taken as a threshold function: if the output of the last fully connected layer < 0 then the input sample belongs to the class 0 else the input sample belongs to class 1.

I'm tempted to say "change your net to 2 outputs and a softmax" but I assume there is a strong reason to not do it this way.

The use of two network outputs instead of one leads to an unnecessary over-parameterization of the task.

[Przemek D]

I understand that you are regressing something like a probability that the object belongs to some class. The accuracy layer has not been designed to work with this sort of classification, but you can still make it work. Your net outputs a probability value of the sample belonging to the class (say, 0.45), but the Accuracy layer expects a vector of (at least) two probabilities: in your case, probability of belonging to the class, and probability of not belonging to it. So you should convert the [0.45] vector into a [0.45, 0.55] one. Here's how I'd do it:

1. Use DummyData Layer to create a vector of [1] values (length equal to the number of samples in your batch)

2. Use Eltwise to subtract the vector of calculated probabilities from 1's

3. Use Concat to concatenate the probabilities with their complements - this gives you the [0.45, 0.55] vectors

This concatenated vector can be fed to the Accuracy layer directly. This will let you calculate the accuracy without overparametrizing the net.