Workstream, number threads

[Praveen C]

Dear all

I use WorkStream in my code. I initialize like this

unsigned int n_threads = 1;

Utilities::MPI::MPI_InitFinalize mpi_initialization (argc, argv, n_threads);

But when I run this code, each process is using 1700% of CPU. The code does not progress in its iterations also. What am I doing wrong ?

Thanks

praveen

[Daniel Arndt]

Praveen,

unsigned int n_threads = 1;

Utilities::MPI::MPI_InitFinalize mpi_initialization (argc, argv, n_threads);

But when I run this code, each process is using 1700% of CPU. The code does not progress in its iterations also. What am I doing wrong ?

This looks correct. Can you verify that you don't have that problem when you are running step-9 (for WorkStream) and step-40 (for MPI)?

Do you have a minimal working example that shows this problem?

Best,

Daniel

[Praveen C]

Hello Daniel

I recompiled dealii without threads.

When I run step-40 single process

./step-40

I see upto 2000% cpu usage during cycle 7 when viewed in top.

Best

praveen

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Martin Kronbichler]

Hi Praveen,

Do you happen to use a BLAS that uses threading? I recommend switching to non-threaded BLAS (or "export OMP_NUM_THREADS=1"). deal.II applies threading on a higher level and a threaded BLAS is counter-productive in 98% of cases.

Best,
Martin

To unsubscribe from this group and stop receiving emails from it, send an email to dealii+un...@googlegroups.com.

[Praveen C]

Hello Martin

I am using openblas installed using spack. openmp seems to be disabled

openblas@0.2.19%gcc@6~openmp+pic+shared arch=linux-opensuse20161217-x86_64

Using

export OMP_NUM_THREADS=1

does the trick. Now I see about 100% cpu for each process.

Thanks a lot

praveen

PS: I also disabled hyperthreading in the bios. Is there any benefit of using this ?

[Denis Davydov]

On Thursday, December 29, 2016 at 11:43:15 AM UTC+1, Praveen C wrote:

Hello Martin

I am using openblas installed using spack. openmp seems to be disabled

openblas@0.2.19%gcc@6~openmp+pic+shared arch=linux-opensuse20161217-x86_64

I think the problem was that Openblas by default builds with USE_THREAD=1 which uses pthreads

thus you saw 1700% usage. 

I will fix this in Spack to have a finer control over threading.

Regards,

Denis.

[Denis Davydov]

A bit off topic: if one is not careful and mixes multithreaded Blas/Lapack + multithreading via TBB in deal.II,

then calls to Lapack's invert() can take 1.371e+03s instead of 1.445e+00s ;-)

p.s. threading in Spack's Openblas and friends is being reworked here https://github.com/LLNL/spack/pull/5340 

Regards,

Denis.

[Praveen C]

I was using FullMatrix.gauss_jordan() in my code. I do this in each cell to solve a local problem in the HDG method. Maybe this was why I observed huge cpu usage and very slow run. After setting OMP_NUM_THREADS=1, it works fine.

Thanks
praveen