Purpose of NUM_THREADS in build process / bug?

[Jeffrey Green]

Hello!

What is the purpose of specifying the NUM_THREADS in Makefile.rule?  

I have 4 real cores on my machine.  When NUM_THREADS is unspecified, MAX_CPU_NUMBER is set to 4 during the build (HT is off).  This makes sense.

I have a multithreaded program that uses OpenBLAS only for sgemm.  When I run it with 4 threads and OPENBLAS_NUM_THREADS=0, everything works perfectly.  If however I try to run 16 threads with OPENBLAS_NUM_THREADS=0, the program crashes with:

BLAS : Program is Terminated. Because you tried to allocate too many memory regions.

This seems like a bug.  If I rebuild OpenBLAS with USE_THREADS=0 and do not modify NUM_THREADS, the 16 thread program also crashes which also seems to be a bug.  

If I rebuild OpenBLAS with MAX_CPU_NUMBER set to 32, the 16 thread program works fine (also using OPENBLAS_NUM_THREADS=0).  

Why then should I not set NUM_THREADS to a really high value to avoid the crash with a high number of threads?  What is the purpose NUM_THREADS when OPENBLAS_NUM_THREADS=0?  

I've tried this with both the v0.2.13 tag and with 771b18ae9c3aa70f4a2497b59700e06819a6ae30.  

Thanks.

[Zhang Xianyi]

Hi Jeffrey,

By default, the NUM_THREADS is equal to the number of cores on your building machine. Thus, it is a limit of threading with this library build. We don't want the user create more threads than the number of cores.

If you want more numbers of threads than the cores of building machine, you should set NUM_THREADS at Makefile.rule.

Xianyi

--
You received this message because you are subscribed to the Google Groups "OpenBLAS-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openblas-user...@googlegroups.com.
To post to this group, send email to openbla...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jeffrey Green]

Hi Xianyi.  Thank you for your response. 

Why is a limit placed on the number of threads though?  Is there any benefit?  Why cap the user at all?  

[Zhang Xianyi]

OpenBLAS allocates somethings likes  temp[NUM_THREADS] on stack or malloc on runtime.

We assume only one thread per core.

Xianyi

[Jeffrey Green]

That makes sense.  Thanks!