Which GPU properly support torch.CudaHalfTensor? (fp16 computation)

[Ken Geonmin Kim]

Hi all 

I am interested in using half-precision computation for GPU.

As I know, torch.CudaHalfTensor() serves fp16 computation.

I tested to make variable torch.CudaHalfTensor(5,2) in several GPUs  

   - GTX980 

   - GTX1080 Ti

   - Titan x

and all of the machines works to create it.

(All of the machines have cuda version >= 7.5)

But I am not sure they perform (for speed & memory) better than torch.CudaTensor() for all GPUs.

Which GPUs properly support usage of torch.CudaHalfTensor?

[fchouteau]

Nvidia seems to have heavily restricted FP16 in consumer Pascal Consumer's GPU so that it does not harm their profesionnal GPU line.

See here for benchmarks: http://www.anandtech.com/show/10325/the-nvidia-geforce-gtx-1080-and-1070-founders-edition-review/5

I have the feeling that FP16 gains will rather be optained on the new P40/P100 GPU than on consumer GP104.