Java Vector and Foreign Memory

[will...@gmail.com]

Peter

Wanted to see if you have done any testing or thoughts on using the new Java Vector API which promises parallel speed up if underlying architecture supports it. https://blogs.oracle.com/javamagazine/java-champion-favorite-java16-records-vector-arm64-github#anchor_2

Similar thoughts on the Foreign memory API which looks like you can now get a block of memory and work with it directly. 

Listened to an interview with the head of JVM Engineering at Oracle and looks like the work they have been doing over the last 5 years to modernize Java in particular for deep learning is starting to get traction. 

Thanks

Scooter

[Peter A]

Hi Scooter,

At a high level I've been monitoring its progress and will probably do some experiments with image convolution, thresholding, and matrix multiplication. One issue is how to integrate this new library in. The API and bytecode version BoofCV targets will always be stuck lagging behind by a few years.  What I think can be done is to hijack the C++ hooks which were added a bit ago and create a new project that the user can optionally load.

I am curious how much of an improvement this will generate. A lot of BoofCV's code has been designed to make it easy for hotspot to vectorize. The main reason things like image convolution aren't faster is that the JVM isn't nearly as good at vectoring as GCC. I've been wanting Oracle to invest more effort in that part of JVM for years.

Anyways, if this does offer a large speed up for code which is already highly optimized the result could be BoofCV absolutely dominating benchmarks like this https://boofcv.org/index.php?title=Performance:OpenCV:BoofCV

Cheers,

- Peter

--
You received this message because you are subscribed to the Google Groups "BoofCV" group.
To unsubscribe from this group and stop receiving emails from it, send an email to boofcv+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/a5428f92-5a8f-48fe-bad9-7fc93860b530n%40googlegroups.com.

--

"Now, now my good man, this is no time for making enemies."    — Voltaire (1694-1778), on his deathbed in response to a priest asking that he renounce Satan.

[Scooter Willis]

Definitely worth benchmarking to see how good they are doing by comparison to what it can figure our from a loop. I suspect the big win will be when they build in GPU support for NVIDIA, AMD, Intel in particular if they can work in the same memory space via foreign memory API. With NVIDIA trying to buy ARM going to assume they plan on putting an ARM core on each or some number of GPU cores. When they did the math it was cheaper to buy ARM. This is when the JVM approach of JIT will leave static compilation behind. 

I went looking for a Java Vector API benchmark and couldn't find any. If you can come up with a particular operation of BoofCV to use as a comparison benchmark.  In particular to help the JVM Vector team optimize for your problem space. I suspect it will also attract the interest of those who understand why this puts Java back in the image processing game and bring deserved attention to BoofCV. Have always been an advocate that if you write good code in Java no reason it should be slow minus the safeguards. Your work in BoofCV clearly shows how an API should perform. 

You received this message because you are subscribed to a topic in the Google Groups "BoofCV" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/boofcv/ZfGYlP2zkKo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to boofcv+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/CAFKAwfspGszr39vVf700kmMnPqKAFt8zE93TG1008K4YGpGp1A%40mail.gmail.com.

[Scooter Willis]

For your performance benchmark on ARM you mention not having an optimized JVM in 2019. Using JDK 15 from https://bell-sw.com/ and performs well. 

[Peter A]

The way I worded it isn't the best. The JVM provided by default wasn't optimized for ARM. That was a mistake by the distribution's maintainers. I found a JVM which was optimized for ARM and provided a link to it. They have since fixed that mistake.

To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/CAMjFWEOGnPQ_fk0dfpZxr4q6WZZKRFEnnWKCZW6hPdVPXHcXQQ%40mail.gmail.com.

[Peter A]

Just tried to do a test and spinning my wheels trying to get past the Module system.  The easiest way is to probably add a command line argument to javac for accessing the incubator API. I'm using gradle and can't get that to work. I added module support with a file and got access the vector API but then lost access to anything that wasn't a Module jar. If there's a way to just disable Modules or to tell it to give me access to everything without restriction this would be easy.

On Sun, Mar 21, 2021 at 11:56 AM Scooter Willis <will...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/CAMjFWEOGnPQ_fk0dfpZxr4q6WZZKRFEnnWKCZW6hPdVPXHcXQQ%40mail.gmail.com.

[Scooter Willis]

Did you download JDK 16? 

If so I think it requires on the command line --enable-preview to make that new module available for use. The default is to hide the preview/incubator classes from the general user. 

To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/CAFKAwfuhi88x_Oxar0z-dJKnn1wpGbsuDhApBtGUauy%3DVuB_mQ%40mail.gmail.com.

[Peter A]

This appears to be a bit of a mess. I think they locked down the JDK even more in 16. Gradle has issues with 16, but in theory I include a hack that should make it work. Adding a module-info.java file just seems to swap one problem for another.

https://github.com/lessthanoptimal/VectorPerformance/blob/master/build.gradle

if you want to give it a shot. The goal is to get it to successfully compile and run the unit test.

https://github.com/gradle/gradle/issues/15538

To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/CAMjFWEPXiL6PzitqO1C7e4APHB_H-eJyJu_E2ZJnLhrKOQ3kmA%40mail.gmail.com.

[Scooter Willis]

I was working with the preview feature in another project and unleashed some conflicts with other libraries so I retreated to a clean project. 

Will do some testing tomorrow. 

To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/CAFKAwfvXcnBwimtEbHGXftk1-imBDc5XMF%2BiDFM3ysN9oO67Gg%40mail.gmail.com.

[Peter A]

I'm now using a milestone of Gradle 7 and with the other hacks seems to be working. Debugging vector code now. I did stumble in a BLAS benchmark published by SPARK on Github. Looks like they saw 2x speed boost.

To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/CAMjFWEOOpz%3DaJ-j-4r%2BGf1VDp2%3DJsMGBL_X5MdzDvZwhUyPbww%40mail.gmail.com.

[Peter A]

Posted results for matrix multiplication. That's about 2x faster for a medium sized matrix. I'll try to get image convolution in there tomorrow.

https://github.com/lessthanoptimal/VectorPerformance