MNIST example with images from IMAGE_DATA and no other changes fails to train

[Pastafarianist]

Hi guys,

I'm trying to train a standard LeNet on MNIST with images loaded from IMAGE_DATA, not DATA (that's because I actually need to train it on smaller digits, about 15x15px, so I want to make sure that it works correctly before plugging in new_width and new_height). Unfortunately, it doesn't work as expected. Training loss rapidly becomes about 10^-9 while testing loss is in the range of 50-60. These would be the perfect signs of overfitting if it wasn't for the fact that this is the exact same LeNet as in the tutorials. Here's the code:

Creating images: https://gist.github.com/7b67e4db822f0e3211b8

Solver: https://gist.github.com/e3669ab569d782e03778

Network: https://gist.github.com/5aa0e615ee14a7d70112

I would *guess* that caffe trains the network on only one batch, which is 100 images but I have no idea how to check that. Could somebody help?

[Pastafarianist]

I confirm that if I replace IMAGE_DATA layers with DATA and an lmdb database generated from the same directory with images, everything works perfectly. The problem is definitely somewhere within IMAGE_DATA.

Here is the updated loader: https://gist.github.com/a0489f705982ebc7f5a2