Regression (sort of) with dynamically generated training examples

[Pastafarianist]

I am working on a real-world character recognition problem. I'd like to take an image of a string of digits and, well, turn it into a string. The standard approach of segmentation based on connected components followed by HOG+SVM has failed due to poor image quality, so I am exploring whether a neural network would work better. I have a bunch of training examples, but far too few to run any sensible training. So I'm generating more examples artificially. Specifically, I generate a string of MNIST digits which looks like the ones I'm trying to recognize. Then I run a sliding window across it, generating a few dozens of patches. To each patch I assign an array of probabilities. The patches that sit directly on top of digits get probability 1 in that digit's class and 0 in others. In other patches I interpolate probabilities between the two nearest digits.

Thus, I'd like to train the network on images that are generated on-the-fly and have an array of class probabilities assigned to them, not a single class ID. I am doing generation in Python.

1. As far as I know, MemoryData layer is the only (currently officially available) way to train the network from Python, and it accepts class IDs as labels, not arrays. Is that correct?
   
2. As a crude but working solution, I could avoid the problem of lacking Python interface by storing a few hundred thousand generated training examples in a database instead of generating them on-the-fly and running the usual caffe binary. But convert_imageset expects class IDs, not arrays. If (1) is correct, how do I generate a database?
3. Would SigmoidCrossEntropyLoss be a good choice for the final layer?
4. Stepping a bit aside, is this a good approach to character recognition?

I would be extremely grateful if someone could help me with some of these questions.

[Axel Angel]

(1) This may be of interest: http://nbviewer.ipython.org/github/BVLC/caffe/blob/tutorial/examples/01-learning-lenet.ipynb

(2) I wrote my own Python program to generate datasets in LMDB, so I can train them with caffe. I like having a dataset on-disk so I can open it easily. You can find examples how to write LMDB in other threads.

(4) I can suggest this paper: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf