Run network forward on single sample, while keeping training batch size large.

[Fredrik Skeppstedt]

Hi,

My name is Fredrik and I'm working on using deep reinforcement learning for autonomous driving for my master thesis.

As a first quick prototype I'm trying to use DQN to teach an agent to keep the right distance to the car in front in a simple simulator.

I would like to keep training in batches, but at some points I need to run the network forward on batches less than the set batch size (typically a size of one sample). 

My question is:

Is there a way to run the network forward in "single sample mode", i.e. with blobs with just one row (of size (1, x, x, x)), while keeping the batch size property in the prototxt file at something like 64 for training? I'm using pycaffe.

Or do I need to have two networks that are identical apart from the batch size and then synchronize the parameter blobs of the 1-batch-size network with the 64-batch-size network after each step of training?

[Jan C Peters]

After loading your network into python with net = caffe.Net(...) you can just resize the input blobs (often called "data" and "label") with net.blobs["data"].reshape(1, x, x, x) to a batchsize of 1, keeping the other dims equal, then load some data into them by doing e. g. net.blobs["data"].data[...] = <your data>. When you execute net.forward() the other blobs in your network will be resized to the correct size automatically. So no, you don't have to change your prototxt file for that.

Jan

[Jan C Peters]

To clarify: You need a "deploy" kind of network prototxt for that, though (i. e. without data layers but instead defining the input blobs with "input" and their shapes with "input_shape" (which can be changed after loading, as I said)). Otherwise the "forward" will overwrite whatever you have done about the input blobs by reading from the Data layers and putting that information in there.

[Fredrik Skeppstedt]

Thank you for your reply Jan!

Do you know where I can find more documentation about deploy networks and the input layer? Can't see it in the tutorial (http://caffe.berkeleyvision.org/tutorial/). 

Can these be used with the solver to train the network as well?

[Jan C Peters]

Actually I had to figure that out the hard way, too. Didn't find much information about that either. Well, this is what I "inferred" by using caffe for quite some time now (the terminology I use is mostly not canon):

In principle, there are two different kinds of network configs, the "train"/"normal" config and the "deploy" config. The train config is used with the solver to train the network, while the deploy config is used to "manually" feed data through the already trained network, i. e. data that is not saved in an appropriate DB/ not specified in a data layer. These two configs are mostly the same, but for the Data layers: The train config contains data layers to fill the blobs (usually called "data" and "label") that are fed into the remaining network, while the deploy config contains no data layers. But caffe still needs to know what the input blobs are and their shape, so instead of data layers the deploy prototxt has to include the "input" and "input_shape" specifiers to inform caffe about where to expect the inputs to the network and how they are shaped, e. g. like it is shown in https://github.com/BVLC/caffe/blob/master/examples/mnist/lenet.prototxt. However, after loading into python the network the shapes can still be changed.

So you can create a deploy version of a net config automatically if you know the dimensions of your data, which you should. I don't know if there are any official scripts for that, I wrote my own.

And no, you can't use deploy configs with a solver.

Jan

[Fredrik Skeppstedt]

Thank you very much for the detailed answer!

Very helpful!