GPU to CPU model converstion

[Lei]

Hi,

Is there any way I can convert cudnn network or cunn network to nn network (GPU to CPU)?

Any examples will be greatly appreciated.

Thanks,

[Jonghoon Jin]

You can run

model_on_cpu = model_on_gpu:float()

to make a copy of the object from GPU to CPU

​

--
You received this message because you are subscribed to the Google Groups "torch7" group.
To unsubscribe from this group and stop receiving emails from it, send an email to torch7+un...@googlegroups.com.
To post to this group, send email to tor...@googlegroups.com.
Visit this group at http://groups.google.com/group/torch7.
For more options, visit https://groups.google.com/d/optout.

[Lei]

Thanks Jin.

The thing is: if I convert the GPU model to CPU model and save that model.

When I try to load the saved CPU model on a machine without GPU, the program has errors:

 "unknown Torch class <torch.CudaTensor>"

I guess the reason is the model is not fully converted and still expects input from CudaTensor?

Is there anyway to fix this?

[Michael Partheil]

I would also be very interested in this! I guess the reason it does not work as expected is that torch.save includes the typenames of the network, and when later on trying to load the network it can't find, say, cudnn.SpatialConvolution or the other CUDA related classes.

A work-around might be to manually save and restore the network weights but this is rather hacky...

[Michael Partheil]

Here is a hacky way that works for me, when using cudnn modules:

function replaceModules(net, orig_class_name, replacer)

  local nodes, container_nodes = net:findModules(orig_class_name)

  for i = 1, #nodes do

    for j = 1, #(container_nodes[i].modules) do

      if container_nodes[i].modules[j] == nodes[i] then

        local orig_mod = container_nodes[i].modules[j]

        container_nodes[i].modules[j] = replacer(orig_mod)

      end

    end

  end

end

function cudnnNetToCpu(net)

  local net_cpu = net:clone():float()

  replaceModules(net_cpu, 'cudnn.SpatialConvolution', 

    function(orig_mod)

      local cpu_mod = nn.SpatialConvolutionMM(orig_mod.nInputPlane, orig_mod.nOutputPlane,

          orig_mod.kW, orig_mod.kH, orig_mod.dW, orig_mod.dH, orig_mod.padW, orig_mod.padH)

      cpu_mod.weight:copy(orig_mod.weight)

      cpu_mod.bias:copy(orig_mod.bias)

      return cpu_mod

    end)

  replaceModules(net_cpu, 'cudnn.ReLU', function() return nn.ReLU() end)

  return net_cpu

end

torch.save('cpu_net.bin', cudnnNetToCpu(gpu_net))

This will replace cudnn.SpatialConvolution and cudnn.ReLU with their CPU equivalents. The saved model can be loaded on a computer without cuda. Of course, if you use other cudnn modules that do not get converted by float() (like cudnn.Softmax) you need to add that to the code above. Probably same if you use fbcunn modules (haven't tested this though).

[Sourav Bhattacharya]

Thanks Michael for the code snippet. I am having one issue with cudnn.SpatialConvolution to nn.SpatialConvolution conversion with the suggested approach. In my case cudnn has self.groups = 2, which is making the self.weight matrix in nn.SpatialConvolution smaller by a factor of 2. Is there any easy fix?

[ignorant]

Update: This is now resolved in https://github.com/soumith/cudnn.torch/pull/76

[Jon Pi]

Hey

Did anyone get a 

attempt to call method 'double' (a nil value)

or

attempt to call method 'float' (a nil value)

when trying:

model = torch.load(modelName)
model = model.float()    # model = model.double()

Thanks!

[Jon Pi]

oops! I meant that when trying:

model = torch.load(modelName)
model = model:float()    # model = model:double()

[Mika S]

Can you check what you are loading is actually an nn module