How to use fp16?

[Ruixin Zhang]

Hi guys,

     I found fp16 is supported in cutorch. Is there some way to use fp16?

     https://github.com/torch/cutorch/commit/8120786180a1d41fca863dbe33fca7e11c988a7e?diff=unified

     https://github.com/soumith/convnet-benchmarks 

[Adam Paszke]

There are fp16 Tensors in cutorch already, but unfortunately there are no math/nn fp16 operations implemented at the moment.

--
You received this message because you are subscribed to the Google Groups "torch7" group.
To unsubscribe from this group and stop receiving emails from it, send an email to torch7+un...@googlegroups.com.
To post to this group, send email to tor...@googlegroups.com.
Visit this group at https://groups.google.com/group/torch7.
For more options, visit https://groups.google.com/d/optout.

[Greg Heinrich]

You might want to help test https://github.com/soumith/cudnn.torch/pull/170.

On Tuesday, April 19, 2016 at 11:55:10 AM UTC+2, Adam Paszke wrote:

There are fp16 Tensors in cutorch already, but unfortunately there are no math/nn fp16 operations implemented at the moment.

[Ganesh Venkatesh]

What is the status now on FP16 support in Torch? Is it supported in convolutions and/or other nn operations? 

Thanks

Ganesh

On Tuesday, April 19, 2016 at 2:55:10 AM UTC-7, Adam Paszke wrote:

There are fp16 Tensors in cutorch already, but unfortunately there are no math/nn fp16 operations implemented at the moment.

[Sergey Zagoruyko]

Training and inference in fp16 are supported with cudnn, I have an example here https://github.com/szagoruyko/wide-residual-networks/tree/fp16

[Ganesh Venkatesh]

Thank you for the pointer. It is very helpful. 

Thanks

-Ganesh

On Saturday, August 20, 2016 at 7:20:41 AM UTC-7, Sergey Zagoruyko wrote:

Training and inference in fp16 are supported with cudnn, I have an example here https://github.com/szagoruyko/wide-residual-networks/tree/fp16

[Chris Padwick]

Hi Sergey.  I am new to torch and am trying to ramp up on how to convert an existing model to use FP16.  I have a Tegra TX1.  I am familiar with the CudaHalfTensor, and I have a small piece of code working as follows:

require 'cutorch'

n = 1000

m = 10000

ht1 = torch.rand(n,m):clone():type('torch.CudaHalfTensor')

ht2 = torch.rand(n,m):clone():type('torch.CudaHalfTensor')

nruns = 1000

start=os.clock()

for i=1,nruns do

  ht3 = ht1:cmul(ht2)

end

stoptime=os.clock()

This appears to me to be working with FP16 instructions because it runs faster than CudaTensor which I believe is a float32 type.  I cloned your repository, but I can't find any reference to CudaHalfTensor, or hasHalf.  I am probably missing something critical here but I can't figure out how the models are using FP16.  Can you point me to the area of code where FP16 is enabled?  Naively I would have thought that if the input to the network is CudaHalfTensor, then the computations would be carried out has FP16.  I can't find anywhere in your code where the input is being changed, so perhaps my postulate is incorrect.

my ultimate goal is to take alexnet.lua and run float32 and fp16 and compare the timings.  I found soumith's repo convnet benchmark repo (https://github.com/soumith/convnet-benchmarks) however it requires x86_64 and won't build on arm64.

thank you!

Chris