Accuracy and loss show different trend

[jin...@gmail.com]

Hello everyone!

I am running the FCN network on my own dataset.

Here is a learning curve figure from my output. This looks really wired for me because the validation loss shows a trend for overfitting. But the mean accuracy appears to be quite stable. I tried to change the learning rate, but it didn't really help.

I am puzzled by this question for a long time. Does anyone know the reason?

Thank you!

Jin

[kishen suraj P]

Won't it depend on the validation dataset you use?

Increasing loss means unstable training.High learning rates perhaps.

What is the type of loss layer you used?

[jin...@gmail.com]

Hi kishen,

Thank you for your reply.

I also think there is something wrong with the training. What I don't understand is why the accuracy didn't decrease while validation loss increased, since high loss means bad performance on the validation dataset.

I tried to change the validation dataset, but got still similar results.

When I decreased the learning rate, the loss unstability got less obvious. It totally disappeared when the learning rate was decreased to 10^-11 or 10^-12, but the accuracy decreased correspondingly. 

I am using the softmax loss layer.

在 2017年2月12日星期日 UTC+9上午2:41:19，kishen suraj P写道：

[kishen suraj P]

On what dataset is accuracy being computed ,Is it validation?

[jin...@gmail.com]

Yes. The accuracy and loss are computed on the same validation set.

I was thinking if I made any mistakes. But I used the original script for computing loss and accuracy, and didn't modify it. So I was confused why they are different.

在 2017年2月12日星期日 UTC+9下午3:57:19，kishen suraj P写道：

[kishen suraj P]

I doubt that your model might be making predictions which are progressively tending to be constant.

Check that once.

[jin...@gmail.com]

I've checked the output. But actually it's making the right predictions, just as the accuracy indicated. Sometimes the accuracy and output is quite good, but the loss is going pretty high. I don't known why it's behaving like this...

在 2017年2月13日星期一 UTC+9上午4:18:28，kishen suraj P写道：