How can I reduce the over-fitting of caffe-reference model ?

[Erogol]

I have 18 different classes with almost 1200 images per each. As I train the model it gives the test set accuracy 35% but after some epochs it starts to reduce to 21%, although loss values is decreasing continuously. I guess this is a sign of over-fitting. If my assumption is true, how should I reduce the over-fitting effect ?

[Prasanna Gyawali]

If you're on Caffe, you can use DropOut Layer for mitigating the problem of overfitting!!

[Kun Duan]

Try reduce the learning rate? you can test and see if your model gives close to 100% accuracy on training set. Also early stop when the test scores do not improve?

[npit]

If you can't get more training examples, then either try a different network architecture or stop when the test accuracy consistently reduces.