GPU decoder PR

[Daniel Povey]

If anyone is interested to try it out, Nvidia is now ready to make public their pull request for the GPU decoder:

https://github.com/kaldi-asr/kaldi/pull/3114

The basic interface is similar to the online2-wav-nnet3-latgen-faster decoder.  

I.e. you decode as if for online-nnet3.

But it's geared towards batch operation (since things get very complicated if you genuinely have streams of data coming in in real time).

Dan

[Tae-young Jo]

Hello, Dan.
It is very good to hear this GPU decoder PR.
However what it means by it is geared toward batch operation? Is it different anyway from the existing nnet 3 batch decoder with GPU?
They utilize GPU in wFST computation?
Thanks in advance.

Best,
Lucas Jo

[Daniel Povey]

The GPU does the graph search stuff too.  In the existing nnet3 batch decoder, the GPU is only used for nnet inference (and the computation time is dominated by the graph search).

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
To post to this group, send email to kaldi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/7399ecce-841a-402b-bd81-f0f89c478791%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tae-young Jo]

Oh then it means this PR utilize GPU in both nnet3 and graph search. It decodes audio in online sense but still is geared toward batch operation...
I’d like to look into this! :)
Thank you for sharing the news

[Phil]

Are there plans to decouple the cuda lattice decoder and nnet3, so users can access the cuda decoding with a similar interface to https://github.com/kaldi-asr/kaldi/blob/master/src/bin/latgen-faster-mapped.cc, so it can operate on features produced by other nnet libraries like pytorch?

If not, could you recommend some places to make changes to get this to work? 

Thanks Dan & NVIDIA team, this is really cool stuff 

[jian kang]

Hi Phil,

It looks more difficult to implement this interface than previous branch. 

I think you can change the structure within "BatchedThreadedCudaDecoder" class.

I am doing some experiments about this but seems more difficult.

Phil於 2019年3月19日星期二 UTC+8上午8時15分01秒寫道：

[Daniel Povey]

I think they have plans to make a version that can work with things like PyTorch.  I'm not sure how toolkit-specific it will be.

--

Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
To post to this group, send email to kaldi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/86ad544b-cdef-4018-bb08-40087b4792a9%40googlegroups.com.

[reza ali]

Hi

I see some GPU PRs recently.

Is it possible to use GPU decoding for codes like TCP port decoding? 

In your opinion, Is it possible to used GPU to decoding waves one by one?

[Justin Luitjens]

Do you have to do waves one by one?  Why can't you batch them.  You can use CUDA to do this but it wouldn't be extremely efficient. 

--

Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
To post to this group, send email to kaldi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/e08f5513-15eb-437d-b92e-d33916c4dd31%40googlegroups.com.

[reza ali]

Based on tcp decoding, models don't delete from memory and can request to decode waves one by one and keep time to load again model in to memory.
My question is is it possible to use GPU decoding in tcp code?
Best regards