Unable to mpirun step-17 and step-18

[Ananya Bijaya]

Hi al,

I have installed Dealii 9.4.2 using "spack install dealii"  on Ubuntu 20.04 inside WSL. When I try to mpirun step-17 and step-18 I either encounter an error message (attached at the end) when the number of cores is more than two or for two cores the memory consumption is huge. In my case, the entire 48 gb of my laptop is getting used.

From the online discussions, I gather this is because of METIS installation. How do I fix it?

Regard,
Ananya

Error Message ( I am copying the output for 1 process)

Cycle 0:
   Number of active cells:       64
   Current memory used:         384 bytes
   Maximum memory used:         384 bytes
***Memory allocation failed for SetupCtrl: maxvwgt. Requested size: 103079215112 bytes

----------------------------------------------------
Exception on processing:

An error occurred in line <170> of file </tmp/ananyabijaya7/spack-stage/spack-stage-dealii-9.4.2-irc2qlg52x23mgbv5gh72ix435j35f4b/spack-src/source/lac/sparsity_tools.cc> in function
    void dealii::SparsityTools::{anonymous}::partition_metis(const dealii::SparsityPattern&, const std::vector<unsigned int>&, unsigned int, std::vector<unsigned int>&)
The violated condition was:
    ierr == 1
Additional information:
    An error with error number -3 occurred while calling a METIS function

Stacktrace:
-----------
#0  /home/ananyabijaya7/spack/opt/spack/linux-ubuntu20.04-skylake/gcc-10.5.0/dealii-9.4.2-irc2qlg52x23mgbv5gh72ix435j35f4b/lib/libdeal_II.g.so.9.4.2:
#1  /home/ananyabijaya7/spack/opt/spack/linux-ubuntu20.04-skylake/gcc-10.5.0/dealii-9.4.2-irc2qlg52x23mgbv5gh72ix435j35f4b/lib/libdeal_II.g.so.9.4.2: dealii::SparsityTools::partition(dealii::SparsityPattern const&, std::vector<unsigned int, std::allocator<unsigned int> > const&, unsigned int, std::vector<unsigned int, std::allocator<unsigned int> >&, dealii::SparsityTools::Partitioner)
#2  /home/ananyabijaya7/spack/opt/spack/linux-ubuntu20.04-skylake/gcc-10.5.0/dealii-9.4.2-irc2qlg52x23mgbv5gh72ix435j35f4b/lib/libdeal_II.g.so.9.4.2: void dealii::GridTools::partition_triangulation<2, 2>(unsigned int, std::vector<unsigned int, std::allocator<unsigned int> > const&, dealii::SparsityPattern const&, dealii::Triangulation<2, 2>&, dealii::SparsityTools::Partitioner)
#3  /home/ananyabijaya7/spack/opt/spack/linux-ubuntu20.04-skylake/gcc-10.5.0/dealii-9.4.2-irc2qlg52x23mgbv5gh72ix435j35f4b/lib/libdeal_II.g.so.9.4.2: void dealii::GridTools::partition_triangulation<2, 2>(unsigned int, std::vector<unsigned int, std::allocator<unsigned int> > const&, dealii::Triangulation<2, 2>&, dealii::SparsityTools::Partitioner)
#4  /home/ananyabijaya7/spack/opt/spack/linux-ubuntu20.04-skylake/gcc-10.5.0/dealii-9.4.2-irc2qlg52x23mgbv5gh72ix435j35f4b/lib/libdeal_II.g.so.9.4.2: void dealii::GridTools::partition_triangulation<2, 2>(unsigned int, dealii::Triangulation<2, 2>&, dealii::SparsityTools::Partitioner)
#5  ./step-17: Step17::ElasticProblem<2>::setup_system()
#6  ./step-17: Step17::ElasticProblem<2>::run()
#7  ./step-17: main
--------------------------------------------------------

Aborting!
----------------------------------------------------

--------------------------------------------------------
An error occurred in line <170> of file </tmp/ananyabijaya7/spack-stage/spack-stage-dealii-9.4.2-irc2qlg52x23mgbv5gh72ix435j35f4b/spack-src/source/lac/sparsity_tools.cc> in function
    void dealii::SparsityTools::{anonymous}::partition_metis(const dealii::SparsityPattern&, const std::vector<unsigned int>&, unsigned int, std::vector<unsigned int>&)
The violated condition was:
    ierr == 1
Additional information:
    An error with error number -3 occurred while calling a METIS function

----------------------------------------------------
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[25959,1],0]
  Exit code:    1
----------------------------------

[Wolfgang Bangerth]

On 9/12/23 23:29, Ananya Bijaya wrote:
>
> I have installed Dealii 9.4.2 using "spack install dealii"  on Ubuntu 20.04
> inside WSL. When I try to mpirun step-17 and step-18 I either encounter an
> error message (attached at the end) when the number of cores is more than two
> or for two cores the memory consumption is huge. In my case, the entire 48 gb
> of my laptop is getting used.
>
> From the online discussions, I gather this is because of METIS installation.
> How do I fix it?

Ananya,
I must admit that I don't know. The error code -3 corresponds to
METIS_ERROR_MEMORY = -3, /*!< Returned due to insufficient memory */
which is perhaps not surprising given that something is trying to allocate 103
TB of memory.

I tried step-17, and it works on my system. So this must be something specific
to Windows. How exactly are you calling the program? With MPI via mpirun? I
don't have experience with running MPI on Windows, but would have assumed that
this is generally not a great idea.

Best
Wolfgang

--
------------------------------------------------------------------------
Wolfgang Bangerth email: bang...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

[Ananya Bijaya]

Wolfgang,

I am running  "cmake ." and "make", followed by "mpirun -np 4 ./step-17 " in the step directory.
Mpirun is working on step-40 and step-42, but not on step-17 and step-18.

I have also installed Dealii 9.5 using Candii. In this installation I am able to mpirun step-17
and step-18. Though I was preferring the spack installation because it installs,  by default,  most of
the dependencies required for running Dealii tutorials.

Thanks,
Ananya

[Wolfgang Bangerth]

Ananya:

> I am running  "cmake ." and "make", followed by "mpirun -np 4 ./step-17 " in
> the step directory.
> Mpirun is working on step-40 and step-42, but not on step-17 and step-18.

That makes sense, because steps 40 and 42 do not use METIS.

> I have also installed Dealii 9.5 using Candii. In this installation I am able
> to mpirun step-17
> and step-18. Though I was preferring the spack installation because it
> installs,  by default,  most of
> the dependencies required for running Dealii tutorials.

I don't know what to suggest. If spack miscompiles METIS, you cannot use it.
It is, fortunately, not widely used in deal.II -- only the
parallel::shared::Triangulation class uses it.

Best
W.

[Ananya Bijaya]

Thanks, Wolfgang.

I can follow the technique used in step-42 for parallel implementation.

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/-ogX1Bi243M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/3bcc779c-501f-cffc-1053-9178d4a56faa%40colostate.edu.