cudnn.VolumetricFullConvolution ?

[Alexander Weiss]

Any chance a cudnn version of VolumetricFullConvolution is in the works?

[Alexander Weiss]

I just wanted to raise this question again.   Is there any chance a cudnn.VolumetricFullConvolution module is being developed or already exists in somebody's fork of cudnn?  I currently have a neural net with two nn.VolumetricFullConvolution layers.  With those two layers in place, the net is 1.2 GB.  With those two layers removed, the network is 41 MB.  That's pretty ridiculous.

[soumith]

That does seem pretty ridiculous.

I think there isn't anyone developing cudnn.VolumetricFullConvolution. I would've if i had some time.

However, if you see the implementation, you can simply copy over the SpatialFullConvolution.lua in cudnn and modify the code slightly to get this working. The changes should be very little.

--
You received this message because you are subscribed to the Google Groups "torch7" group.
To unsubscribe from this group and stop receiving emails from it, send an email to torch7+unsubscribe@googlegroups.com.
To post to this group, send email to tor...@googlegroups.com.
Visit this group at https://groups.google.com/group/torch7.
For more options, visit https://groups.google.com/d/optout.

[Alexander Weiss]

Okay, thanks for the reply.  I'll see if I can get it working on my own.

On Thursday, September 29, 2016 at 1:27:34 PM UTC-4, smth chntla wrote:

That does seem pretty ridiculous.

I think there isn't anyone developing cudnn.VolumetricFullConvolution. I would've if i had some time.

However, if you see the implementation, you can simply copy over the SpatialFullConvolution.lua in cudnn and modify the code slightly to get this working. The changes should be very little.

On Thu, Sep 29, 2016 at 11:23 AM, Alexander Weiss via torch7 <torch7+APn2wQcsaR8bcOeppHpjYu30PfpJTwn3luLMBUUyH0EVaRyRp1q7UJHGY@googlegroups.com> wrote:

I just wanted to raise this question again.   Is there any chance a cudnn.VolumetricFullConvolution module is being developed or already exists in somebody's fork of cudnn?  I currently have a neural net with two nn.VolumetricFullConvolution layers.  With those two layers in place, the net is 1.2 GB.  With those two layers removed, the network is 41 MB.  That's pretty ridiculous.

On Tuesday, September 13, 2016 at 4:27:53 PM UTC-4, Alexander Weiss wrote:

Any chance a cudnn version of VolumetricFullConvolution is in the works?

--
You received this message because you are subscribed to the Google Groups "torch7" group.

To unsubscribe from this group and stop receiving emails from it, send an email to torch7+un...@googlegroups.com.

[Alexander Weiss]

I copied over cudnn/SpatialFullConvolution.lua to cudnn/VolumetricFullConvolution.lua and carefully modified it, line by line.  I also added VolumetricFullConvolution to init.lua.  However, when I try to forward a 5D tensor through cudnn.VolumetricFullConvolution, I get the following error:

/home/ubuntu/torch/install/share/lua/5.1/cudnn/init.lua:118: Error in CuDNN: CUDNN_STATUS_BAD_PARAM (cudnnGetConvolutionForwardAlgorithm)
stack traceback:
    [C]: in function 'error'
    /home/ubuntu/torch/install/share/lua/5.1/cudnn/init.lua:118: in function 'errcheck'
    ...nstall/share/lua/5.1/cudnn/VolumetricFullConvolution.lua:181: in function 'createIODescriptors'
    ...nstall/share/lua/5.1/cudnn/VolumetricFullConvolution.lua:315: in function 'forward'
    [string "out = net:forward(img)"]:1: in main chunk
    [C]: in function 'xpcall'
    /home/ubuntu/torch/install/share/lua/5.1/trepl/init.lua:670: in function 'repl'
    ...untu/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:199: in main chunk
    [C]: at 0x00406670

Line 181 in cudnn/VolumetricFullConvolution.lua is:

errcheck('cudnnGetConvolutionForwardAlgorithm',
   cudnn.getHandle(),
   self.oDesc[0], self.weightDesc[0],
   self.convDesc[0], self.iDesc[0],
   algSearchMode, algWorkspaceLimit, algType)

Any guesses on what I might have messed up?  (I don't understand much of what's going on in these cudnn lua files, I just naively changed all references of "Spatial" to "Volumetric", added "kT", "dT", "padT" and "adjT" in obvious places, and changed indices as needed to deal with 5D tensors instead of 4D tensors.)

[soumith]

Can you point me to your partial implementation? I am happy to debug / complete it.

On Thu, Sep 29, 2016 at 11:23 AM, Alexander Weiss via torch7 <torch7+APn2wQcsaR8bcOeppHpjYu30PfpJTwn3luLMBUUyH0EVaRyRp1q7UJ...@googlegroups.com> wrote:

I just wanted to raise this question again.   Is there any chance a cudnn.VolumetricFullConvolution module is being developed or already exists in somebody's fork of cudnn?  I currently have a neural net with two nn.VolumetricFullConvolution layers.  With those two layers in place, the net is 1.2 GB.  With those two layers removed, the network is 41 MB.  That's pretty ridiculous.

On Tuesday, September 13, 2016 at 4:27:53 PM UTC-4, Alexander Weiss wrote:

Any chance a cudnn version of VolumetricFullConvolution is in the works?

--
You received this message because you are subscribed to the Google Groups "torch7" group.
To unsubscribe from this group and stop receiving emails from it, send an email to torch7+un...@googlegroups.com.
To post to this group, send email to tor...@googlegroups.com.
Visit this group at https://groups.google.com/group/torch7.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "torch7" group.

To unsubscribe from this group and stop receiving emails from it, send an email to torch7+unsubscribe@googlegroups.com.

[Alexander Weiss]

Hi Soumith.  I sent you an email with cudnn/VolumetricFullConvolution.lua attached.  Thanks!

On Thu, Sep 29, 2016 at 5:08 PM, torch7 on behalf of smth chntla <tor...@googlegroups.com> wrote:

Can you point me to your partial implementation? I am happy to debug / complete it.

--
You received this message because you are subscribed to a topic in the Google Groups "torch7" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/torch7/gaYdEqAiO6Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to torch7+unsubscribe@googlegroups.com.

[soumith]

A working version of VolumetricFullConvolution is now merged into master via https://github.com/soumith/cudnn.torch/pull/266

--

S

On Thu, Sep 29, 2016 at 6:59 PM, torch7 on behalf of Alexander Weiss <tor...@googlegroups.com> wrote:

Hi Soumith.  I sent you an email with cudnn/VolumetricFullConvolution.lua attached.  Thanks!

On Thu, Sep 29, 2016 at 5:08 PM, torch7 on behalf of smth chntla <tor...@googlegroups.com> wrote:

Can you point me to your partial implementation? I am happy to debug / complete it.

[Alexander Weiss]

Thanks for getting that done!  It should have a big impact on my network.

On Thursday, September 29, 2016 at 9:10:30 PM UTC-4, smth chntla wrote:

A working version of VolumetricFullConvolution is now merged into master via https://github.com/soumith/cudnn.torch/pull/266

--

S

On Thu, Sep 29, 2016 at 6:59 PM, torch7 on behalf of Alexander Weiss <tor...@googlegroups.com> wrote:

Hi Soumith.  I sent you an email with cudnn/VolumetricFullConvolution.lua attached.  Thanks!

On Thu, Sep 29, 2016 at 5:08 PM, torch7 on behalf of smth chntla <tor...@googlegroups.com> wrote:

Can you point me to your partial implementation? I am happy to debug / complete it.

--

You received this message because you are subscribed to a topic in the Google Groups "torch7" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/torch7/gaYdEqAiO6Y/unsubscribe.

To unsubscribe from this group and all its topics, send an email to torch7+un...@googlegroups.com.

To post to this group, send email to tor...@googlegroups.com.
Visit this group at https://groups.google.com/group/torch7.
For more options, visit https://groups.google.com/d/optout.

[Alexander Weiss]

Just thought I would follow up with some feedback now that cudnn.VolumetricFullConvolution is implemented:

My serialized neural net is now 41 MB (as compared to 1.2 GB previously)

My training speed has improved by ~30%.