TestGradient

[Tho Mas]

Hi.

I was writing a caffe layer on cpu as well on gpu. However TestGradient does not stop running when testing the GPU Backward method. 

[==========] Running 8 tests from 4 test cases.
[----------] Global test environment set-up.
[----------] 2 tests from MyLayerTest/1, where TypeParam = caffe::CPUDevice<double>
[ RUN      ] MyLayerTest/1.TestForward
[       OK ] MyLayerTest/1.TestForward (883 ms)
[ RUN      ] MyLayerTest/1.TestGradient
[       OK ] MyLayerTest/1.TestGradient (843 ms)
[----------] 2 tests from MyLayerTest/1 (1727 ms total)


[----------] 2 tests from MyLayerTest/3, where TypeParam = caffe::GPUDevice<double>
[ RUN      ] MyLayerTest/3.TestForward
[       OK ] MyLayerTest/3.TestForward (124 ms)
[ RUN      ] MyLayerTest/3.TestGradient

Any help appreciated.

[Tho Mas]

Calling the Backward_gpu method works well. Only using the GradientChecker is an issue...

[Tho Mas]

Okay, i just figured out that the GradientChecker just takes a pretty long while to process... My 3 bottom blobs are of size 512 x 3 x 2 x 1 and the top blob is of size 1. Does it always take that long on GPU?

[Przemek D]

Gradient tests are usually quite tedious, because the numeric checker does not run on the GPU. Your layer is executed on the GPU, but the reference values that its outputs are compared to are computed on the CPU - that's why it takes so long.

The recommended approach is to test layers on minimal blobs, 20-100 elements suffice.