Tools for understanding GPUs?

[William Pietri]

Years of coding, performance tuning, and ops work have given me some level of mechanical sympathy, albeit nothing like many of the contributors to this list. But now dealing with GPUs I feel like a noob again. 8704 cores! And I feel like I'm peering through a keyhole trying to figure out what's going on in there.

These days I'm working on the Anti-Defamation League's Online Hate Index. We are creating ML models to spot many kinds of hate, and we aim to apply them at scale to high-volume sources like social media. Luckily, ML experts are handling the modeling. I'm mainly working on the "at scale" part. As a non-profit, optimizing for cost is especially important to us, so I really need to understand the performance.

What tools and resources do folks here like for developing their intuitions about GPU activity? And if you have ML-specific resources, even better.

Thanks,

William

[Henri Tremblay]

I learned playing with Aparapi (https://aparapi.com/).

It's not super complicated but you need to jump right in mechanical sympathy with GPU. To decide vector length and type of memory used. It makes a huge difference.

I would love to tell you to try Java 16 and the Vector API but it doesn't seem to support GPU. Just AVX. That's my understanding at least.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/mechanical-sympathy/895470ac-05fc-c2f5-526f-66b33d5cd65d%40scissor.com.

[Remi Forax]

De: "Henri Tremblay" <henri.t...@gmail.com>
À: "mechanical-sympathy" <mechanica...@googlegroups.com>
Envoyé: Lundi 22 Mars 2021 12:17:23
Objet: Re: Tools for understanding GPUs?

I learned playing with Aparapi (https://aparapi.com/).

It's not super complicated but you need to jump right in mechanical sympathy with GPU. To decide vector length and type of memory used. It makes a huge difference.

I would love to tell you to try Java 16 and the Vector API but it doesn't seem to support GPU. Just AVX. That's my understanding at least.

yes,

The Vector API support AVX, AVX-2 and AVX-512 and also NEON but do not support GPUs.

Rémi

On Sun, 21 Mar 2021 at 14:36, William Pietri <wil...@scissor.com> wrote:

Years of coding, performance tuning, and ops work have given me some level of mechanical sympathy, albeit nothing like many of the contributors to this list. But now dealing with GPUs I feel like a noob again. 8704 cores! And I feel like I'm peering through a keyhole trying to figure out what's going on in there.

These days I'm working on the Anti-Defamation League's Online Hate Index. We are creating ML models to spot many kinds of hate, and we aim to apply them at scale to high-volume sources like social media. Luckily, ML experts are handling the modeling. I'm mainly working on the "at scale" part. As a non-profit, optimizing for cost is especially important to us, so I really need to understand the performance.

What tools and resources do folks here like for developing their intuitions about GPU activity? And if you have ML-specific resources, even better.

Thanks,

William

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/mechanical-sympathy/895470ac-05fc-c2f5-526f-66b33d5cd65d%40scissor.com.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/mechanical-sympathy/CADZL2%3DtNxpPjTCx6v5V5kgmWG09H2g2y_ffk-%2Bmq5N3K8Nmm%2BQ%40mail.gmail.com.

[Ben Evans]

Might TornadoVM be of use here?

https://www.infoq.com/presentations/tornadovm-java-gpu-fpga/

https://www.infoq.com/articles/tornadovm-java-gpu-fpga/

If Dr Fumero is not on this list, I can introduce you to him by email.

Thanks,

Ben