Multithreading in BEAST2

[RC]

Dear List,

I'm trying to understand the multithreading options in BEAST2 a little better.

I have a 250 (taxon) x 6,000 (site) alignment with two partitions, and separate site models for each partition. Partition A has 3,000 unique site patterns, and partition B has 1,500 unique site patterns.

Using the 'ThreadedTreeLikelihood' option I divided partition A into four (threads="4"), and partition B into two (threads="2"). Now there are six data chunks of 750 unique site patterns each.

I run BEAST2 with the following flags '-beagle_CPU -threads 6 -beagle_instances 6'. Now I understand that this is different to BEAST1, where beagle_instances would divide up each of the two major partitions six ways.

The thing is, I get only a 10% performance increase using six threads rather than one (default). Did I set this up incorrectly, or is 750 unique patterns too few for one thread and am I therefore better off running multiple independent runs using only a single CPU for each?

Thanks for any advice.

[Remco Bouckaert]

Did you set useThreads=true on the element with id=likelihood? If not, the two likelihoods will be calculated sequentially.

If you set the flag on the likelihood, you may want to increase the number of threads to start BEAST with, since both the ThreadedTreeLikelihood and CompoundDistribution use threads from the pool. With '-threads 8’ there should be sufficient threads in the pool.

The beagle instances option does not work (logged as issue #138).

You might want to try the -beagle_SSE flag instead of the -beagle_CPU flag, and squeeze a bit more speed out of BEAGLE.

Cheers, Remco

--
You received this message because you are subscribed to the Google Groups "beast-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beast-users...@googlegroups.com.
To post to this group, send email to beast...@googlegroups.com.
Visit this group at http://groups.google.com/group/beast-users.
For more options, visit https://groups.google.com/d/optout.

[RC]

Thanks Remco, that's really helpful.

I followed the guide at beast2.org/wiki/index.php/Performance_Suggestions but there's no mention there of adding 'useThreads=true' to the id=likelihood element.

Have added this now, and am running with all 8 threads (was actually using SSE, but simplified my explanation a little). The threads are now "flatlining" at 70-80%, while before they were oscillating wildly, so it's definitely had an effect.

Still not getting much of a speedup though. Only 10% faster than with one thread. Suggests to me the bottleneck is elsewhere? The tree topology is fixed by the way, and the clock model is the RLC.

Cheers

[Remco Bouckaert]

On 10/05/2015, at 6:35 am, RC <rupert...@gmail.com> wrote:

Thanks Remco, that's really helpful.

I followed the guide at beast2.org/wiki/index.php/Performance_Suggestions but there's no mention there of adding 'useThreads=true' to the id=likelihood element.

Perhaps you missed the part on CompoundDistribution due to its title — I changed the title of that section.

Have added this now, and am running with all 8 threads (was actually using SSE, but simplified my explanation a little). The threads are now "flatlining" at 70-80%, while before they were oscillating wildly, so it's definitely had an effect.

Still not getting much of a speedup though. Only 10% faster than with one thread. Suggests to me the bottleneck is elsewhere? The tree topology is fixed by the way, and the clock model is the RLC.

It is possible that threading overhead is higher than the gain you get by using threads. You could try reducing the number of threads for the ThreadedTreeLikliehoods and see whether that helps.

Cheers, 

Remco