0.003 difference in AlexNet Top-1, Top-5 accuracy on ILSVRC2012

[Stephanie Wang]

I'm getting a small difference in validation accuracy from the reported Top-1 and Top-5 reported: https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet

I get Top-1 = 0.56828, Top-5  = 0.799501

As opposed to the reported Top-1 = .571, top-5 = .802

I create the LMDB cropping 256x256, download the imagemean binary.proto, and checked that I am validating for 1000 iterations.

Can anyone provide insight on what is causing this 0.003 difference? 

[Evan Shelhamer]

Hey Stephanie,

This difference in metrics is not so large, but I would expect that we can make it zero, since I trained and evaluated the BVLC AlexNet. That said, little changes in data preparation or computation can compound.

- What kind of interpolation did you use for making the 256x256 images? I would have to go back into some (years old) logs to know, but I believe I picked bilinear interpolation.

- What backend are you running, cuDNN or vanilla Caffe? You could try doing inference with the vanilla Caffe backend by setting `engine: CAFFE` in the proto definition, or by compiling Caffe without cuDNN.

I would suspect a difference in the data more than a difference in the computation, since cuDNN was initially developed to pass all the Caffe unit tests, but one never knows.

Hope that helps,

Evan Shelhamer

--
You received this message because you are subscribed to the Google Groups "Caffe Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to caffe-users+unsubscribe@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/3d7a6ba9-6a7d-4a48-8909-2eb28da8b34b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.