CUDA KERNEL ERROR while running e2e scripts

[Acharya KLN]

Dear All,

While training data for an e2e system, I'm receiving the following error.

cudaError_t 209 : "no kernel image is available for execution on the device" returned from 'cudaGetLastError()'

I have searched the forums but I couldn't find any solution. I also check the kaldi.mk file for compatible architectures but even those were matching. 

Attached are the nvidia-smi output, training log and kaldi.mk file for the error.

Any help is highly appreciated.

Thank you,

Acharya.

[Daniel Povey]

search the forum for that error message, it was asked before.

--
Go to http://kaldi-asr.org/forums.html to find out how to join the kaldi-help group
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/1b338cd7-9577-4b5b-9882-cce1507f80ddn%40googlegroups.com.

[Acharya KLN]

Hi Dan,

I tried the solution mentioned before in the forum. but we have not been able to rectify the error.

Any other solution available?

You received this message because you are subscribed to a topic in the Google Groups "kaldi-help" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kaldi-help/Fs6s3v3404w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kaldi-help+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/CAEWAuyQ3d2j9MGXdo16jx_P%2BnNr%2BKMwfadqnG466f5qSzLWJHw%40mail.gmail.com.

[Daniel Povey]

Maybe you didn't do "make clean" before making

Dependencies on the makefile are not tracked

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/CADGbKdxK1_gMEN2X%2BbFNNM9gmJxr5%2BRgSzTNOwNFsWTbWB6uoQ%40mail.gmail.com.

[Hugo Braun]

Hi Acharya,

In your kaldi.mk your CUDA_ARCH is not valid. I guess you tried to use the --cuda-arch opt for ./configure ? No need to use that option unless you want to speed up compilation. Just use the default and it should be fine.

Hugo

[Acharya KLN]

Hi Hugo,

Thank you for the reply. I shall try and recompile kaldi.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/4ed06c1c-3c7b-47f9-889f-54b2031dc562n%40googlegroups.com.