cuDNN: same network, varying runtimes

[Bart Vosteen]

I build a ResNet-34 clone and did some manual timing testing, with both CAFFE and CUDNN as engine for the convolution part. The timing results i obtained with CAFFE are consistent with measurements i made some weeks ago. With CUDNN as engine it takes now nearly double the time in comparison.

Does cuDNN performance depend on specific kernel and library versions? Or are there some "hidden" configuration files that I did forget about?

Thanks for any ideas.


My system:
Driver Version: 375.39
cuDNN Version: 5.1
CUDA Version: 8.0
Ubuntu 16.04.2 LTS
Kernel: 4.4.0-62-generic

[Detlef Schmicker]

if you have a gtx 970 it has 3.5 gb fast memory and 0.5 gb slow, making a lot of difference if the net is in the slow as you need more than 3.5 gb

[Bart Vosteen]

I use a GTX 1060 6Gb. To my knowledge all its memory is attached with the same bandwidth.

Just to make it clear, the only thing that changed, was the point in time i took the measurements.