Overfitting on resnet 50 or other ?

[webuc...@gmail.com]

I'm using a resnet 50 network to train my model from scratch using my dataset.

As we can see on the graph the loss validation is growing. I think it is overfitting .

I have a DB about 75k image with 55 categories (around 1k images per cat) and 25% validation images.

I use 256x256 resize images with squash transformation. In the network it is mirroring already on training images.

I plan to add more data but what could I do to have more data augmentation ?

Any advice is welcomed.

Kind regards,

[Jon Watte]

Yes, when the validation loss is noticeably higher than the training loss, you are quite likely overfitting. Your graph looks like a textbook example of this :-)

To avoid overfitting, you can do a few things:

1) Use fewer parameters. Fewer layers, smaller layers.

2) Use more training material. Come up with more training data. Make sure it's meaningfully distinct. Perhaps save less for validation? (10%?)

3) Use response normalization, such as LRN layers. (This sometimes helps and sometimes not in my experience.)

4) Add dropout layers. Drop out at least 10% of data from the first perceptive layers; drop out at least 50% from the later decisive/analytic layers.

How much dropout is enough? When your validation loss is lower than your training loss, you don't need to add more dropout.

However, that may not always be realistic, so some experimentation may be required.

Sincerely,

jw

Sincerely,

Jon Watte


--
"I find that the harder I work, the more luck I seem to have." -- Thomas Jefferson

--
You received this message because you are subscribed to the Google Groups "DIGITS Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to digits-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/digits-users/48aa24ff-4ab8-4986-9a17-48bc068b0f09%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Shah Nawaz]

Can you please share prototxt?