In Mex with MATLAB, OMP threads start but do not share work

[Chris Lengerich]

Hi,
I'm having issues trying to use OpenMP within a mex file. Strangely, it seems that threads are being started, but that they can't see each other (and so each one thinks it is singlethreaded). As a result, they don't share work and the mex file runs much slower than the single threaded version. However, when executing the same program on the system,

On MATLAB R2008a, I have a simple mex file which I am compiling in gcc 4.4.1. I get the following output after setting OMP_NUM_THREADS to 4 in the shell and running from within MATLAB:

Hello World from thread = 0
Number of processors: 8
In parallel? 0
Number of threads = 1
Hello World from thread = 0
Number of processors: 8
In parallel? 0
Number of threads = 1
Hello World from thread = 0
Number of processors: 8
In parallel? 0
Number of threads = 1
Hello World from thread = 0
Number of processors: 8
In parallel? 0
Number of threads = 1

Running same binary in the shell, I get:

Hello World from thread = 2
Hello World from thread = 1
Hello World from thread = 0
Hello World from thread = 3
Number of processors: 8
Number of processors: 8
In parallel? 1
In parallel? 1
Number of processors: 8
In parallel? 1
Number of processors: 8
In parallel? 1
Number of threads = 4

So it seems that the mexa is creating 4 threads but they don't know about each other (and therefore won't share work). The program compiles fine and runs multithreaded on R2009a (which incidentally has the Parallel Programming Toolkit installed). Has anyone else ever had this problem?

Please find source code and compile calls below:

///////////////////
Program
//////////////////
#include <omp.h>
#include "mex.h"

void mexFunction( int nlhs, mxArray *plhs[],
int nrhs, const mxArray *prhs[] )
{

int nthreads, tid;
#pragma omp parallel private(nthreads, tid)
{
tid = omp_get_thread_num();
printf("Hello World from thread = %d\n", tid);
printf("Number of processors: %d\n", omp_get_num_procs());
printf("In parallel? %d\n", omp_in_parallel());
if (tid == 0)
{
nthreads = omp_get_num_threads();
printf("Number of threads = %d\n", nthreads);
}
}
}

///////////////////////////
Compile calls for mexa64:
///////////////////////////
$GCC/g++ -c -L$GOMP -Wl,-rpath,$GOMP -I/usr/local/matlab/extern/include -DMATLAB_MEX_FILE -ansi -D_GNU_SOURCE -fopenmp -fPIC -fno
-omit-frame-pointer -pthread -DMX_COMPAT_32 -O -DNDEBUG test.cpp

$GCC/gcc -c -L$GOMP -Wl,-rpath,$GOMP -I/usr/local/matlab/extern/include -DMATLAB_MEX_FILE -ansi -D_GNU_SOURCE -fopenmp -fexceptions -fPIC -fno-omit-frame-pointer -pthread -DMX_COMPAT_32 -O -DNDEBUG "/usr/local/matlab/extern/src/mexversion.c"

$GCC/g++ -O -pthread -shared -L$GOMP -Wl,-rpath,$GOMP -Wl,--version-script,/usr/local/matlab/extern/lib/glnxa64/mexFunction.map -Wl,--no-undefined -o "test.mexa64" test.o mexversion.o -Wl,-rpath-link,/usr/local/matlab/bin/glnxa64 -L/usr/local/matlab/bin/glnxa64 -lmx -lmex -lmat -lm -fopenmp

[Oliver Woodford]

Chris, I've had this problem. The mex file ignores the OMP_NUM_THREADS environment variable, using 1 instead, so creating only 1 thread. Try this in the pragma to fix the problem:
#pragma omp parallel num_threads(4) private(nthreads, tid)

In my code I use:
#pragma omp parallel num_threads(omp_get_num_procs()) private(nthreads, tid)
to make sure I have a thread per processor, thus maximizing the available computing power.

HTH,
Oliver

"Chris Lengerich" <chris.l...@gmail.com> wrote in message <hap259$obm$1...@fred.mathworks.com>...

[chris]

Hmm...still no go. Adding the num_threads(4) call to the top doesn't seem to change the pattern. I still have:

Hello World from thread = 0
Number of processors: 8
Hello World from thread = 0
Number of processors: 8
In parallel? 0
Number of threads = 1
Hello World from thread = 0
Number of processors: 8
In parallel? 0
Number of threads = 1
In parallel? 0
Number of threads = 1
Hello World from thread = 0
Number of processors: 8
In parallel? 0
Number of threads = 1

Note: it seems that four threads are being created (the outputs are each called four times and by the cpu usage, I'm using all four processors, spinning them each to 100% (with a little more complex code in the middle.) But still, it seems that the threads aren't aware that they are in parallel. Any thoughts?

"Oliver Woodford" <o.j.woo...@cantab.net> wrote in message <hapl6l$dqv$1...@fred.mathworks.com>...

[chris]

Found a workaround. Hopefully this can help someone:

It turns out that the threads are indeed sharing work, despite showing the same thread number. Why this happens is definitely obscure, but regardless, even with the same thread number it works. What prevented me from seeing this the first time was the fact that maxNumCompThreads was set to 8 (for an eight processor system), which somehow MATLAB ended hogging all of the cpu cycles, and so the program (with the additional of some parallel for loops) ran so slow it seemed not to share work. Setting maxNumCompThreads(1) restricted MATLAB's use of the CPU and allowed OMP to use the processors.

It may be an issue with some aspect of the MATLAB configuration - on other machines, it seems that there is no need for this kind of jerry-rigging, but here it was absolutely necessary.

Anyway, hope this helps.

"Chris Lengerich" <chris.l...@gmail.com> wrote in message <hap259$obm$1...@fred.mathworks.com>...

[Son Hua]

In your M-file, add this before calling your MEX-file.

setenv('OMP_NUM_THREADS', '2'); % utilize dual cores

This will set the environment variable OMP_NUM_THREADS to 2. It works for me under MATLAB R2009a. When it is not set, the MEX-file runs in a single thread. When it is set to 2, it runs on two threads as expected.

Cheers,
Son.