CUDA support when using readNetFromONNX

[Mr. B]

Hey everyone,

I am trying to modify a Java script (see here), which uses dnns readNetFromONNX.

Sadly I am unable to add CUDA support, but on the other hand I am a newbie to Java in general.

To implement CUDA I've tried to:

1. add opencv-platform-gpu and cuda-platform-redist as build-dependencies in gradle
   
2. add this in line #58 from the original script:
   
   net.setPreferableBackend(dnn.DNN_BACKEND_CUDA);
   net.setPreferableTarget(dnn.DNN_TARGET_CUDA);
   
   

My try #1 didn't change anything, the load is still on the CPU (and slow). I had some hopes for this, since I followed the sample usage.

The #2 didn't even compile, due to the lack of dnn recognizing DNN_BACKEND_CUDA or DNN_TARGET_CUDA. Also without the "dnn." part, the commands didn't work.

This might have been how OpenCV should have make use of CUDA, see here.

So after multiple hours of unsuccessful trying, I headed to this group:
Does someone can help me, on how to implement CUDA in the script?

Is there a tutorial, showing how to implement the CUDA functions within JavaCV?

Thanks in advance

[Samuel Audet]

This isn't something that is supported well by the current release of OpenCV.

We'll need to wait for the next release of OpenCV that includes the patch,

but you could also help get this pull request merged if you need this right away:

https://github.com/bytedeco/javacpp-presets/pull/832

[Samuel Audet]

OpenCV 4.3.0 has been released with a fix for this:

https://github.com/bytedeco/javacpp-presets/commit/288cc6c04bc1c7e0522e08b539450f0e80512feb

Please give it a try with the snapshots:

http://bytedeco.org/builds/

[Jan-Philipp Buck]

Thanks Samuel, that sounds really good.

Sadly I am stuck at compiling - I’ve tried to compile with Maven on Windows, but run into different problems: if I do it out of the box, ffmpeg is failing due to missing android jars. If I limit the compile to javacpp.platform windows-x86_64, javacv fails due to a missing windows jar in the snapshot.

What am I doing wrong? I am using Visual Studio Code on Windows 10 64-bit, and within the Maven extension to mvn -U compile with a pom for 1.5.3 files

[Samuel Audet]

No need to compile anything, the snapshots contain precompiled binaries: http://bytedeco.org/builds/

[Jan-Philipp Buck]

Thanks Samuel, for the quick response.

I build now succesfully with 1.5.3, since I kept struggling with the Snapshots.

Though my GPU is being used, I do see barely any improvement in terms of FPS.

Which is rather weird, given that a previous Python script I've been using saw a 10x improvement moving from CPU-only to CUDA.

Any clue what I might have done wrong?

I am building with javacv, opencv-platform, opencv-platform-gpu and cuda-platform-redist.

The use of CUDA I try activating via the net.set-commands:

net = readNetFromONNX(modelPath.toAbsolutePath().toString());
net.setPreferableBackend(DNN_BACKEND_CUDA);
net.setPreferableTarget(DNN_TARGET_CUDA);

Am I tackling the problem from the wrong side?

Maybe my implementation is just to simple?

[Samuel Audet]

JavaCPP maps the C++ API, not the Python API, so we cannot easily
compare them. Do you have code in C++ that is working as expected? If
so, please let me know where I can find that code, and then we will be
able to find easily what the difference is with your code in Java.

[Samuel Audet]

Here's something else we could try. Try to execute your Python code with
the binaries from the presets, like this:
https://github.com/bytedeco/javacpp-presets/blob/master/opencv/samples/OpenCVVersion.java

Let me know what kind of performance you get with that. If the
performance is poor, then we'll know there's something wrong with my
binaries, and not your code. Thanks!

Samuel