caffe.set_mode_cpu() still requires GPU?

[Jason Yosinski]

Hi,

I'm trying to use Caffe via the python bindings in CPU mode on a
machine with no GPU.



Case 1:
When I compile caffe with the following options, I can load and use
the library in Python without a GPU:
CPU_ONLY := 1
#USE_CUDNN := 1 # commented out



Case 2:
However, I normally compile caffe with GPU and CUDNN enabled:
# CPU_ONLY := 1 # commented out
USE_CUDNN := 1

When I do this, it seems that even after switching into CPU mode via
caffe.set_mode_cpu(), the library looks for a GPU and then aborts when
it can't find one. For example, here's the procedure I follow on a
machine with no GPU:


<compile as in Case 2 above>
$ python
>>> import caffe

>>> caffe.set_mode_cpu()
FATAL: Module nvidia not found.
WARNING: Logging before InitGoogleLogging() is written to STDERR
E0521 11:58:03.446987 25610 common.cpp:93] Cannot create Cublas
handle. Cublas won't be available.
E0521 11:58:03.447023 25610 common.cpp:94] Error is: 1
FATAL: Module nvidia not found.
E0521 11:58:03.451866 25610 common.cpp:101] Cannot create Curand
generator. Curand won't be available.
E0521 11:58:03.451884 25610 common.cpp:102] Error is: 203
E0521 11:58:03.452370 25610 common.cpp:103] Error is: 101
E0521 11:58:03.452378 25610 common.cpp:107] DeviceQuery:
FATAL: Module nvidia not found.
E0521 11:58:03.457087 25610 common.cpp:161] No cuda device present.
No cuda device present.

>>> net = caffe.Classifier('deploy.prototxt', 'weights_450000.caffemodel')
I0521 11:58:17.952971 25610 net.cpp:42] Initializing net from parameters:
name: "CaffeNet"
input: "data"
input_dim: 1
...
I0521 11:58:17.953521 25610 net.cpp:338] conv1 -> conv1
I0521 11:58:17.953542 25610 net.cpp:113] Setting up conv1
F0521 11:58:17.953726 25610 cudnn_conv_layer.cpp:31] Check failed:
error == cudaSuccess (38 vs. 0) no CUDA-capable device is detected
*** Check failure stack trace: ***
Aborted (core dumped)


Is this expected? Is the only way to prevent looking for a GPU to
re-compile in CPU_ONLY mode?

Thanks,
jason


---------------------------
Jason Yosinski, Cornell Computer Science Ph.D. candidate
http://yosinski.com/ +1.719.440.1357

[Evan Shelhamer]

Is this expected? Is the only way to prevent looking for a GPU to
re-compile in CPU_ONLY mode?

No, CPU_ONLY mode isn't required to run on a machine without a GPU. The Caffe singleton does try to load cuda handles like cublas and curand but it merely complains and moves on if it cannot. (#1500 could settle this by taking mode out of the singleton s.t. GPU initialization is only needed once a GPU net or solver is instantiated.)

The issue here is that with `USE_CUDNN` on the default engine is cuDNN, and the crash is coming from the cuDNN layer:

F0521 11:58:17.953726 25610 cudnn_conv_layer.cpp:31] Check failed:
error == cudaSuccess (38 vs. 0)  no CUDA-capable device is detected

In CPU mode cuDNN layer will fallback on the {forward,backward}_cpu calls of ConvolutionLayer but its LayerSetup() is still called. One could handle this by changing the convolution layer factory to check mode, but then switching to GPU mode will use the vanilla Caffe convolution and not cuDNN...

Another workaround is to explicitly set the `engine: CAFFE` field in the model definition or switch the caffe.proto default.

The heart of the matter is deciding when to set mode and whether it should be mutable at all. Dismantling the singleton and device abstraction would help but both require serious effort.

Evan Shelhamer

[Jason Yosinski]

> In CPU mode cuDNN layer will fallback on the {forward,backward}_cpu calls of
> ConvolutionLayer but its LayerSetup() is still called. One could handle this
> by changing the convolution layer factory to check mode, but then switching
> to GPU mode will use the vanilla Caffe convolution and not cuDNN...
> Another workaround is to explicitly set the `engine: CAFFE` field in the
> model definition or switch the caffe.proto default.

Got it; thanks. I just wanted to make sure I wasn't doing something stupid.

I think a practical workaround for me (and perhaps others wanting to
mix CPU / cuDNN use) will be just to have two separately compiled
copies around.

Thanks,
jason


---------------------------
Jason Yosinski, Cornell Computer Science Ph.D. candidate
http://yosinski.com/ +1.719.440.1357