Training accuracy is lesser than validation accuracy

[Gowdham]

Hi,

I'm running a model on my own dataset to do binary classification. I'm getting my training accuracy lesser than validation accuracy on all epoch. I thought it could be due to Dropout in Fully Connected layers, so I set Dropout(0.) but it didn't work.

Could you please let me know why I'm facing this scenario? and its solution?

Epoch 1/10
3320/3320 [==============================] - 76s - loss: 0.3359 - acc: 0.8611 - val_loss: 0.2075 - val_acc: 0.9233
Epoch 2/10
3320/3320 [==============================] - 69s - loss: 0.1972 - acc: 0.9211 - val_loss: 0.1289 - val_acc: 0.9433
Epoch 3/10
3320/3320 [==============================] - 69s - loss: 0.1320 - acc: 0.9500 - val_loss: 0.0745 - val_acc: 0.9833
Epoch 4/10
3320/3320 [==============================] - 69s - loss: 0.0938 - acc: 0.9675 - val_loss: 0.0661 - val_acc: 0.9800
Epoch 5/10
3320/3320 [==============================] - 69s - loss: 0.0613 - acc: 0.9771 - val_loss: 0.0403 - val_acc: 0.9933
Epoch 6/10
3320/3320 [==============================] - 69s - loss: 0.0453 - acc: 0.9840 - val_loss: 0.0353 - val_acc: 0.9900

Your help is appreciated.Thanks.

[Matias Valdenegro]

Hi,

 

We cannot really tell you why, as we don't have your data or network configurations. As for speculations, this could be due that the train/test set split is not random and the train set is harder than the test set.

 

Try making random splits of your data to make train/test sets and see if there is still such an odd pattern.

 

Also note that the differente is quite small, so a couple of hard train samples could create this bias.

[lvd...@gmail.com]

Also, if you are using dropout, it can cause validation accuracy to be higher than training accuracy.

[Gowdham]

Thank you for your response. 

I shuffled the training and validation data set and it works. Initially, the issue still exists but after few epochs training accuracy is higher than validation. I think its not an issue, please correct me if I'm wrong.

[Daπid]

I have two hypothesis: one is that your test at its easier than the training, or there is some bias. The other is that acc is averaged over the whole epoch, while val_acc is taken at the end, when the model is better. You can check this evaluating after each epoch on your training set. 

--
You received this message because you are subscribed to the Google Groups "Keras-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keras-users+unsubscribe@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/keras-users/7e02af58-6dbe-403b-8b91-93fd93c7a3ff%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.