Different accuracy between Theano and Caffe on the same model and dataset

[T Nguyen]

Hi! 

Recently, I trained a network which has the architecture similar architecture to GoogleNet to classify sample images into 3 categories. I trained the network on both Theano and Caffe and I realize that there is a gap between accuracies produced by 2 libraries. 

Details: 
Class     |   0   |   1   |   2   

Theano   |   1%  |   1% | 3%
Caffe      |    0% |   1%  |  11%

The training data set has 5000 images each class. The train/ val data ratio as set at 4:1.

What do you think the reason for the diference? 

[par...@unlv.nevada.edu]

      Did you find this after averaging over multiple runs of the test ?

[Przemek D]

Train the same network on Caffe twice and you'll find different accuracies, because training is in itself a stochastic process. Three sources of randomness that I know of are:

• explicitly random network initialization
  
• picking random order of training examples during training
• nondeterministic implementation of backward passes for some layers in cuDNN (which I think can be turned off)