image processing: best image size to use for training?

[Mathias Rauen]

Let's suppose I have a convnet which does some image processing (e.g. noise reduction), and let's say for simplicity sake it consists of 20 conv+relu layers, each having a kernel size of 3.

It seems for training such a convnet, Caffe wants to get images with a size of 41x41 pixels or bigger. I have seen several Caffe projects where big images were cut into (for this example) 41x41 sized image crops and then stored into a HDF5 file.

Let's say I have a hundred training images in the size 4100x4100. Now I'm wondering is it really the best approach to cut these images into 41x41 sized chunks for training? Or is it "better" to feed the full sized 4100x4100 images into training? I know, I could just test what works better. But it's not as easy to test as it sounds, because it seems to vary depending on the number/configuration of the layers, and I'm not sure how to compare both approaches in a "fair" way (same amount of training etc).

So basically my question is: What are the implications of feeding 41x41 images into this convnet, or 4100x4100 images? How does Caffe handle either case? Where are the advantages and disadvantages of either approach?

I'm thinking, when using 41x41 crops with a 41 stride, for each crop only really the center pixel is they key for each training pass. But when using 4100x4100 images, probably training considers all the pixels, but enforces equal weights. That makes me think that using 4100x4100 images should be better in theory, unless I use a sliding window for the 41x41 crops, which slides only 1 pixel per crop, so one 4100x4100 image would give me 16483600 41x41 crops (if I calculated correctly), which sounds insane. But in real life testing it seems 41x41 crops work very well, maybe even better than 4100x4100 images. But I'm totally confused about why that would be the case, and which image size I should use for optimal training.

Thanks!