How would you like your parallel linear algebra ?

121 views
Skip to first unread message

Clement Pernet

unread,
Mar 22, 2019, 6:15:04 AM3/22/19
to sage-devel
Hi,

In https://trac.sagemath.org/ticket/27444 we are exposing the parallel versions (using OpenMP) of
fflas-ffpack routines used in SageMath. This is only about multicore parallelism, based on OpenMP.

We would like to discuss on the best design to expose them to the end user:

1. Should the default behaviour be sequential or parallel ?

2. If the default is parallel, how many cores should be used by default? I guess the default should
be whatever value openmp_get_numthreads() returns, which is usually the number of cores, however, in
most systems where hyperthreading is enabled, this means that we will use twice as many cores as the
number of real cores, which will slow down the computation.

3. What interface do we want for the sage user who wants to

  3a switch between sequential and parallel,

  3b specify the number of cores to be used ?

Should there be some kind of environment variable containing such informations?

Note that passing an extra argument is not (always) an option, as we would like that the * operator
for matrices benefits from this parallization.

In its current status, the branch on #27444 calls parallel fgemm when multiplying two modn_dense
matrices over the maximal number of available OMP threads. It currently possible to use a specified
number of threads by exporting OMP_NUM_THREADS=XXX before lauching sage.

Thanks for your feedback.

Clément

Thierry Dumont

unread,
Mar 22, 2019, 7:43:30 AM3/22/19
to sage-...@googlegroups.com
Hi,

May be what openblas is doing could be inspiring.

Openblas (I hope it did not changed, I have tested this sometime ago)
allows to use the number of thread defined for open_mp (by setting an
environment variable or by program), and also to specify the number of
threads for openblas. This is important (at least for me) when you do
nested parallelism: you have say n threads, each one is using openblas
and thus you dont want openblas to be parallel.

Also, it is possible do get the numer of cpu in the program (but
counting them twice in cas of hyperthreading, and more with exotic
processors (4 threads / cpu with XeonPhi)).

Thierry
tdumont.vcf

Eric Gourgoulhon

unread,
Mar 22, 2019, 9:36:10 AM3/22/19
to sage-devel
Hi,

Regarding your question 3:

Le vendredi 22 mars 2019 11:15:04 UTC+1, Clement Pernet a écrit :

3. What interface do we want for the sage user who wants to

   3a switch between sequential and parallel,

   3b specify the number of cores to be used ?

Should there be some kind of environment variable containing such informations?


note that there exists already in Sage the singleton class Parallelism(), which has been invented precisely for this:
For instance, in your case, it could be used as

sage: Parallelism().set('fflas-ffpack', nproc=8)

or any better name that you may figure out, e.g.

sage: Parallelism().set('linear_algebra', nproc=8)

Note that one can also use the shorten version

sage: Parallelism().set(nproc=8)

to trigger parallel computations in all declared fields, i.e. including fflas-ffpack if you have registered it in the dictionary attribute `_nrpoc` of class Parallelism.

Best wishes,

Eric.

Eric Gourgoulhon

unread,
Mar 22, 2019, 9:45:05 AM3/22/19
to sage-devel

Travis Scrimshaw

unread,
Mar 22, 2019, 7:40:36 PM3/22/19
to sage-devel

In https://trac.sagemath.org/ticket/27444 we are exposing the parallel versions (using OpenMP) of
fflas-ffpack routines used in SageMath. This is only about multicore parallelism, based on OpenMP.

+1 to having multicore linear algebra in Sage. I have hacked together some things in the past to do computations that would take too long serially, but they were really ugly, and I would prefer not to have to do them again.
 
We would like to discuss on the best design to expose them to the end user:

1. Should the default behaviour be sequential or parallel ?

I think it should follow SAGE_NUM_THREADS as a default. This is something that you have to know at least a little bit of what you are doing to set it. Moreover, if you are doing this for building/testing, then you would want to do the same for computations.

2. If the default is parallel, how many cores should be used by default? I guess the default should
be whatever value openmp_get_numthreads() returns, which is usually the number of cores, however, in
most systems where hyperthreading is enabled, this means that we will use twice as many cores as the
number of real cores, which will slow down the computation.

IMO, same as the above: Start with SAGE_NUM_THREADS and allow the user to modify it accordingly.

3. What interface do we want for the sage user who wants to

   3a switch between sequential and parallel,

   3b specify the number of cores to be used ?

Should there be some kind of environment variable containing such informations?

As Eric mentioned, I would expand upon Parallelism() as that would make a good unique and natural point to have all of this data.

Best,
Travis

Reply all
Reply to author
Forward
0 new messages