Breeze, netlib-java, BLAS, and Vector Instructions (SSE, AVX, etc)

[Jongse Park]

Hi, I am a novice Spark user and am trying to understand my Spark cluster's MLLib performance results. 

I investigated many stuff and reached to here, Breeze groups since I realized that the linear algebra computation (in general vectorization) is done by the Breeze library. 

Let me just put my question up front. I wondered if my cluster is using vector instructions such as SSE or AVX instructions. (Of course, my processors support SSE/AVX instructions) 

BLAS I have is the libblas-dev that I installed through apt-get. 

Could you please help me figure out this simple question? I struggled quite a lot but still haven't figured this out..  

[David Hall]

sorry for the slow reply. somehow missed the moderator email.

netlib-java will usually log to let you know if it loaded native libraries. Whether or not the particular blas you installed is compiled with SSE isn't something I know how to answer.

--
You received this message because you are subscribed to the Google Groups "Scala Breeze" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-breeze+unsubscribe@googlegroups.com.
To post to this group, send email to scala-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scala-breeze/d45cf364-9f42-4134-809e-aa64aedd0da2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Debasish Das]

Recently I have pushed native blas through spark and breeze...Idea here comes to compilation of native blas...Mac BLAS (veclib) is cool..but linux blas (default one that comes with CentOS till 7.2 at least) is archaic...Idea is to use a OpenBLAS compilation with all the SSE, MMX and other support that you want...default compilation of OpenBLAS uses them...infact you can put multi-core support in it as well...It worked fine with Breeze proximal.quadraticminimizer as a client mode but I did not see multithreaded runtime gain from spark (it may be due to the way they assign core to partition)...

Let me know what to change on your cluster as a gist when you start seeing native code runtime gain...What I have found is for level-1 operations, scala/java code is pretty (not complicated level-1 I only tried dot) but I believe it is worth experimenting with both level-1 / level-3 blas with openblas before you start writing your own java code...

On Wednesday, November 2, 2016 at 6:03:48 PM UTC-7, David Hall wrote:

sorry for the slow reply. somehow missed the moderator email.

netlib-java will usually log to let you know if it loaded native libraries. Whether or not the particular blas you installed is compiled with SSE isn't something I know how to answer.

On Mon, Oct 31, 2016 at 10:01 PM, Jongse Park <parkj...@gmail.com> wrote:

Hi, I am a novice Spark user and am trying to understand my Spark cluster's MLLib performance results. 

I investigated many stuff and reached to here, Breeze groups since I realized that the linear algebra computation (in general vectorization) is done by the Breeze library. 

Let me just put my question up front. I wondered if my cluster is using vector instructions such as SSE or AVX instructions. (Of course, my processors support SSE/AVX instructions) 

BLAS I have is the libblas-dev that I installed through apt-get. 

Could you please help me figure out this simple question? I struggled quite a lot but still haven't figured this out..  

--
You received this message because you are subscribed to the Google Groups "Scala Breeze" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-breeze...@googlegroups.com.