Hi all,
I've been running into some pretty significant differences in performance using caffe from a docker image.
Timing `net.forward()` on a net slightly larger than the VGG_ILSVRC_16_layer, I get:
- Mac OS X, gpu mode: 500-1000 ms
- Mac OS X, cpu mode: 1000-2000 ms
- Ubuntu on AWS g2.2xlarge, gpu mode: 100-750 ms
- Ubuntu on AWS g2.2xlarge, cpu mode: 5000 ms (???)
- Docker image, based on nvidia/cuda:7.0-cudnn2-devel-ubuntu14.04, running with nvidia-docker on the g2 instance: 5000 ms
- Same docker image, running with normal docker client in CPU mode on OS X machine: 5000 ms
Any ideas why these last three would show such poor performance?
Given the fact that CPU mode on the ubuntu box WITHOUT docker is performing quite badly, I was wondering if it might have something to do with the linux versions of some dependency like BLAS or something... are there any known issues of that sort?
Thanks in advance,
Anand