How to limit number of threads use with backend OpenMP

[Charly Empereur-mot]

Hello,

I would like to limit the number of threads that MDAnalysis is allowed to use when I specify backend OpenMP.

Are there arguments currently implemented for this ? Or a known work-around that would do the job ?

It is not convenient that all cores are systematically used, for example if we run analysis scripts on clusters.

Thank you very much for any help you might provide !

[Oliver Beckstein]

Hi,

Try setting the OMP_NUM_THREADS environment variable. (OpenMP is implemented via cython without any special considerations http://docs.cython.org/en/latest/src/userguide/parallelism.html.)

Oliver

--

Oliver Beckstein

orbe...@gmail.com * orbe...@gmx.net

[Charly Empereur-mot]

Thank you, this works.

I used:

import os

os.environ['OMP_NUM_THREADS'] = str(nb_threads)

before MDA usage.

[Charly Empereur-mot]

Unfortunately this does not work when I use functions from another python module.

I tried using decorators to set the environment variable each time I call the functions that use OpenMP backend, but that does not help.

If you have other suggestions, I would very much like to ear them !

Otherwise, I imagine these cython calls would just need to receive argument num_threads.

I guess if I Ctrl+F "cython.parallel.prange" in the MDA code, I will have some place to start doing this, right ?

[Oliver Beckstein]

On Oct 21, 2019, at 3:01 PM, Charly Empereur-mot <charly....@gmail.com> wrote:

Unfortunately this does not work when I use functions from another python module.

I tried using decorators to set the environment variable each time I call the functions that use OpenMP backend, but that does not help.

If you have other suggestions, I would very much like to ear them !

Otherwise, I imagine these cython calls would just need to receive argument num_threads.

I guess if I Ctrl+F "cython.parallel.prange" in the MDA code, I will have some place to start doing this, right ?

Not quite so simple. Have a look under MDAnalysis/lib  such as c_distances_openmp.pyx and in particular include/calc_distances.h might be easy to overlook. 

You should also be able to build MDAnalysis without OpenMP by setting

use_openmp = False

in the setup.cfg (I think – I just looked at setup.py and there’s a line 'use_openmp = config.get('use_openmp', default=True)’…)

Oliver

On Monday, October 21, 2019 at 10:23:33 PM UTC+2, Charly Empereur-mot wrote:

Thank you, this works.

I used:

import os

os.environ['OMP_NUM_THREADS'] = str(nb_threads)

before MDA usage.

On Monday, October 21, 2019 at 9:51:46 PM UTC+2, Oliver Beckstein wrote:

Hi,

Try setting the OMP_NUM_THREADS environment variable. (OpenMP is implemented via cython without any special considerations http://docs.cython.org/en/latest/src/userguide/parallelism.html.)

Oliver

On Oct 21, 2019, at 12:48 PM, Charly Empereur-mot <charly....@gmail.com> wrote:

Hello,

I would like to limit the number of threads that MDAnalysis is allowed to use when I specify backend OpenMP.

Are there arguments currently implemented for this ? Or a known work-around that would do the job ?

It is not convenient that all cores are systematically used, for example if we run analysis scripts on clusters.

Thank you very much for any help you might provide !

--

Oliver Beckstein

orbe...@gmail.com * orbe...@gmx.net

--
You received this message because you are subscribed to the Google Groups "MDnalysis discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mdnalysis-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mdnalysis-discussion/7e48d704-5f70-4252-9012-f7344bc4bcde%40googlegroups.com.

[Charly Empereur-mot]

Thank you !

I need to run a lot of simulation and analyze on-the-fly on clusters so the speed-up from OpenMP is almost mandatory, but I cannot have it occupy all the cores.

I would say the pragma need to be fed the number of threads in include/calc_distances.h

line 626 for example

#pragma omp parallel for private(i, rji, rjk, x, xp, y) shared(angles)

needs to be

#pragma omp parallel num_threads(nthreads) for private(i, rji, rjk, x, xp, y) shared(angles)

And then the nthreads argument has to be propagated to API calls. First in c_distances_openmp.pyx it seems, but then also somewhere else ?

Does this seems right ? I might give it a shot

To unsubscribe from this group and stop receiving emails from it, send an email to mdnalysis-discussion+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mdnalysis-discussion/7e48d704-5f70-4252-9012-f7344bc4bcde%40googlegroups.com.

[Johannes Zeman]

Setting the OMP_NUM_THREADS environment variable *before importing* MDAnalysis always worked for me. If you do it after the MDA import, the setting will be ignored.
Having the number of OpenMP threads as an optional kwarg as you suggested would be a neat feature. It should probably have a default value of multiprocessing.cpu_count() or equivalent.

[Richard Gowers]

For what it's worth, I think the openmp stuff in distances is cute but isn't actually that good (I wrote it so I get to bash it :)) - the scaling I get from it was always pretty low, probably because there's usually a lot of extra code around analysis than just distance calculations. I think a much better approach is to parallelise around a wider scope (around a function operating on an AtomGroup), ie what things like pmda and similar (https://github.com/kugupu/kugupu/blob/master/kugupu/generate_results.py#L175) do.

In terms of shoving in the nthreads argument, currently we do a little trick where we use the same header twice, once with a -DPARALLEL.  So you'd have to include the nthreads argument for both versions of the functions and modify both distances.pyx and distances_openmp.pyx to provide this new argument, with the serial version just passing an unused argument.  Then you'd need to check that the dispatcher in distances.py (function _run) is properly passing along the new keyword argument.

To unsubscribe from this group and stop receiving emails from it, send an email to mdnalysis-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mdnalysis-discussion/7e48d704-5f70-4252-9012-f7344bc4bcde%40googlegroups.com.

--

Oliver Beckstein

orbe...@gmail.com * orbe...@gmx.net

--
You received this message because you are subscribed to the Google Groups "MDnalysis discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mdnalysis-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mdnalysis-discussion/054B85D4-2DAE-4ED2-BC2B-1EB76C597A47%40googlemail.com.