Cuda Core kernel in parallel with Tensor Core

Skip to first unread message

aran nokan

Sep 27, 2021, 8:44:25 AM9/27/21
to MAGMA User

Before that I read somewhere that we can not run CUDA kernel in parallel with Tensor core kernel.

But I am seeing sometimes inside of nsys that they have overlap.

So my question is that are they working in parallel or not? If so, why are they not going parallel every time and just for a small kernel? I expected that they were separate parts.

Is this behavior the same in Volta and Ampere?

Do we have any references for understanding more? I did not find a good explanation. 

Best regards,

Ahmad Abdelfattah

Sep 27, 2021, 11:58:51 AM9/27/21
to aran nokan, MAGMA User
As far as I understand, there is nothing that should prevent such an overlap. The utilization of the Tensor Cores is irrelevant. 

Since these question are not specific to MAGMA, I suggest that you check out the CUDA C Programming Guide (


You received this message because you are subscribed to the Google Groups "MAGMA User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
To view this discussion on the web visit

Reply all
Reply to author
0 new messages