Sum of Convolutions fails in GPU mode

[Swami]

I am trying an experiment where the architecture of my network is a smaller version of Alexnet. I have two input layers both of them being RGB images via the ImageData layer and it includes transformation operations like mean subtraction and cropping. The skeleton of the top part of the proto. is given below:

input: data1
layer: conv11 with input data1
input: data2
layer: conv12 with input data2
layer: eltwise sum - inputs: conv11,conv12 & output: conv1_sum
layer: conv2 with input conv1_sum
...
...
...
layer: SoftmaxWithLoss

I would expect this to work right away but strangely the network training proceeds fine in CPU mode but in GPU mode the loss quickly diverges (within 10 or so iterations).

I am puzzled at why something like this should happen. Any help or feedback regarding debugging this will be appreciated.