TBB error inside FESystem constructor when using development version dealii (dealii-8.5.1 works fine) in debug mode

[Sambit Das]

Hello dealii-team,

I have narrowed down my issue to the FESystem constructor in the following simple piece of code (written in main.cc)

<headers>

int main (int argc, char *argv[])

{

  dealii::Utilities::MPI::MPI_InitFinalize mpi_initialization (argc, argv);

  dealii::FESystem<3>  FE1(dealii::FE_Q<3>(dealii::QGaussLobatto<1>(2)), 1);

  return 0;

}

When I run on ddt debugger single processor I get the following error message:

Process 0: Memory error detected in tbb::task::wait​_for​_all ​(task.h:763)​:null pointer dereference or unaligned memory access. I have attached the pic of the stacktrace from ddt. The error is coming from deall/base/thread_management.h:

void join () const

    {

      AssertThrow (joinable(), ExcNoTask());

      task_descriptor->join ();

    }

I am not able to post a minimal example for the above error as it doesn't occur when I compile main.cc standalone, but if I compile main.cc along with other .cc files in my application I get the above error. I suspect that when main.cc is compiled standalone it links to the dealii's bundled tbb, but when it is compiled with other .cc files it links to intel's tbb as seen from the stacktrace. I have checked my dealii installation and it uses bundled tbb. I am using intel/18.0.1, openmpi/3.0.0/intel/18.0.1 and cmake/3.5.2 for dealii installation, with petsc, slepc and p4est as external libraries.

Thanks,

Sambit

[Matthias Maier]

Hi,

On Wed, Mar 21, 2018, at 12:56 CDT, Sambit Das <dsa...@umich.edu> wrote:

> I suspect that when main.cc is compiled standalone it links to the
> dealii's bundled tbb, but when it is compiled with other .cc files it
> links to intel's tbb as seen from the stacktrace. I have checked my
> dealii installation and it uses bundled tbb. I am using intel/18.0.1,
> openmpi/3.0.0/intel/18.0.1 and cmake/3.5.2 for dealii installation,
> with petsc, slepc and p4est as external libraries.

This is exactly the problem! You cannot mix two TBB libraries (internal
and external in your case, but that doesn't matter) in the link
interface.

Out of curiosity, can you please attach the detailed.log file so that we
can have a look at the full link interface? :-) I am curious where tbb
comes in.

You have to recompile deal.II to use the external TBB library
instead. [1]

Best,
Matthias

[1] I checked - on my systems you minimal example does not show a
problem - nice minimal test case by the way!

[Sambit Das]

Hi Matthias,

Thanks for your reply.

Out of curiosity, can you please attach the detailed.log file so that we
can have a look at the full link interface? :-) I am curious where tbb
comes in.

I have attached belowdealiiDetailed.log. There it is picking up the bundled tbb, but when i took a look at the CMakeOutput.log  (attached below dealiiInstallationCMakeOutput.log) - it has picked up intel's tbb.

 

You have to recompile deal.II to use the external TBB library
instead. [1]

Is there a dealii cmake flag to force dealii to use external tbb library?

Best,

Sambit 

[Sambit Das]

Hi Matthias,

I reinstalled dealii to use external tbb library and that resolved my issue :) 

I used the following additional flags:

-DDEAL_II_WITH_THREADS=ON -DTBB_INCLUDE_DIRS="/sw/arcts/centos7/intel/18.1/compilers_and_libraries_2018.1.163/linux/tbb/include/tbb" -DTBB_USER_INCLUDE_DIRS="/sw/arcts/centos7/intel/18.1/compilers_and_libraries_2018.1.163/linux/tbb/include/tbb" -DTBB_LIBRARIES="/sw/arcts/centos7/intel/18.1/compilers_and_libraries_2018.1.163/linux/tbb/lib/intel64_lin/gcc4.4/libtbb.so"

Thanks a lot for your suggestion.

Best,

Sambit

[Matthias Maier]

On Thu, Mar 22, 2018, at 02:50 CDT, Sambit Das <dsa...@umich.edu> wrote:

> Hi Matthias,
>
> I reinstalled dealii to use external tbb library and that resolved my issue
> :)
> I used the following additional flags:
>
> -DDEAL_II_WITH_THREADS=ON
> -DTBB_INCLUDE_DIRS="/sw/arcts/centos7/intel/18.1/compilers_and_libraries_2018.1.163/linux/tbb/include/tbb"
> -DTBB_USER_INCLUDE_DIRS="/sw/arcts/centos7/intel/18.1/compilers_and_libraries_2018.1.163/linux/tbb/include/tbb"
> -DTBB_LIBRARIES="/sw/arcts/centos7/intel/18.1/compilers_and_libraries_2018.1.163/linux/tbb/lib/intel64_lin/gcc4.4/libtbb.so"
>
> Thanks a lot for your suggestion.

Very nice!

That's an awkwardly scattered library :-/ - if everything would be in
one place, you could have simply used -DTBB_DIR=... instead.

Best,
Matthias