Mumps Linking issue

[Chad Balen]

Hi,

don't know if I accidentally posted this multiple times because I got: an error (#354) occurred while communicating with the server. If so sorry

I'm running openSUSE 12.3 (x86_64) and I've been recently trying to install dealii with MUMPS, Trilinos, PETSc, p4est, and Open MPI. I'm installing:
- deal.ii-8.1.0
- Trilinos 11.4.3
- PETSc 3.4.3*
- MUMPS 4.10.0
- p4est 0.3.4.2
*I know you say to use <=3.4.2 but I was having issue installing that version and having dealii recognizing it but it recognized 3.4.3 from YaST easily.

When I finally got it to install I didn't receive any errors so I assumed it worked. To see my deallii install output see the attachment configure_deal.tar.gz. The reason for a tar is because I break it up into error and standard output to make it easier to see and same goes for different files for configure/make/install/test. As well, I attached my build script for installing dealii in attachments (configure_deal.tar.gz).

I then went on to use the steps to see if everything was working properly. The programs for Trilinos and PETSc, seemed to run fine. However I'm having issues with MUMPS.

So when I try to run step-3 and step-4 with MUMPS, I added:

#include "/home/cbalen/Installed_Programs/deal.II/Install/include/deal.II/lac/sparse_direct.h" //For MUMPS

and I changed the solve function like so:

void Step3::solve()
{
  SparseDirectMUMPS  solver;
  solver.initialize (system_matrix, system_rhs);
  solver.vmult(solution, system_rhs);
}

But I get the following when I try to run step-3:
$ cmake ../ #I like to build in a sub-folder
-- The C compiler identification is GNU 4.7.2
-- The CXX compiler identification is GNU 4.7.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Autopilot invoked
###
#
#  Project  step-3  set up with  deal.II-8.1.0  found at
#      /usr/local
#
#  CMAKE_BUILD_TYPE:          Debug
#
#  You can now run
#       $ make                - to compile and link the program
#       $ make run            - to (compile, link and) run the program
#
#       $ make debug          - to switch the build type to 'Debug'
#       $ make release        - to switch the build type to 'Release'
#
#       $ make edit_cache     - to change (cached) configuration variables
#                               and rerun the configure and generate phases of CMake
#
#       $ make strip_comments - to strip the source files in this
#                               directory off the documentation comments
#       $ make clean          - to remove the generated executable as well as
#                               all intermediate compilation files
#       $ make runclean       - to remove all output generated by the program
#       $ make distclean      - to clean the directory from _all_ generated
#                               files (includes clean, runclean and the removal
#                               of the generated build system)
#
#       $ make help           - to view this message again
#
#  Have a nice day!
#
###
-- Configuring done
-- Generating done
-- Build files have been written to: /home/cbalen/Installed_Programs/deal.II/Install/examples/step-3/Mumps/Build

$ make run
Scanning dependencies of target step-3
[100%] Building CXX object CMakeFiles/step-3.dir/step-3.cc.o
Linking CXX executable step-3
CMakeFiles/step-3.dir/step-3.cc.o: In function `Step3::solve()':
/home/cbalen/Installed_Programs/deal.II/Install/examples/step-3/Mumps/step-3.cc:501: undefined reference to `dealii::SparseDirectMUMPS::SparseDirectMUMPS()'
/home/cbalen/Installed_Programs/deal.II/Install/examples/step-3/Mumps/step-3.cc:502: undefined reference to `void dealii::SparseDirectMUMPS::initialize<dealii::SparseMatrix<double> >(dealii::SparseMatrix<double> const&, dealii::Vector<double> const&)'
/home/cbalen/Installed_Programs/deal.II/Install/examples/step-3/Mumps/step-3.cc:503: undefined reference to `dealii::SparseDirectMUMPS::vmult(dealii::Vector<double>&, dealii::Vector<double> const&)'
/home/cbalen/Installed_Programs/deal.II/Install/examples/step-3/Mumps/step-3.cc:503: undefined reference to `dealii::SparseDirectMUMPS::~SparseDirectMUMPS()'
/home/cbalen/Installed_Programs/deal.II/Install/examples/step-3/Mumps/step-3.cc:503: undefined reference to `dealii::SparseDirectMUMPS::~SparseDirectMUMPS()'
collect2: error: ld returned 1 exit status
make[2]: *** [step-3] Error 1
make[1]: *** [CMakeFiles/step-3.dir/all] Error 2
make: *** [all] Error 2

It might be useful to know that to install Mumps I had some issues when following your instructions so I had to make some tweaks:
Issue 1: I would get from Mumps (about Scalapack) a bunch of lines saying:

undefined reference to `pb_topget_' AND
undefined reference to `pb_topset_'
**Exact error message in Mumps_Install.tar.gz under Mumps/topget_and_topset_Error

Thanks to http://dphysique.isc.chubu.ac.jp/Scalapack%20on%20pgi%20&%20RH7.3, I followed there suggestion for compiling these Scalapack functions before installing Scalapack like so:
mpicc -c -fPIC -DAdd_ PB_topget_.c
mpicc -c -fPIC -DAdd_ PB_topset_.c

NOTE: I didn't follow the exact method that Scalapack would have tried to compile them with (i.e mpicc -c -O4 -fPIC -Df77IsF2C -DNO_IEEE -DUsingMpiBlacs -DAdd_ PB_topget_.c) because I still got the same error from MUMPS so I kept what I thought were the important parts.

Issue 2: I then got a bunch of undefined references on libscalapack from Mumps like this person: http://icl.cs.utk.edu/lapack-forum/viewtopic.php?f=2&t=711, I followed their suggestion and added the lapack library to LIBBLAS like so:

LIBBLAS = /usr/lib64/libblas.so /usr/lib64/liblapack.so

If you would like to see my configure scripts, makefiles, Error and Standard Output files for Blacs, Scalapack, and Mumps they are attached (Mumps_Install.tar.gz)

I hope I included everything necessary if I forgot anything let me know. Any help help would be much appreciated.

[Toby Young]

Hello Chad,

> I'm running openSUSE 12.3 (x86_64) and I've been recently trying to install
> dealii with MUMPS, Trilinos, PETSc, p4est, and Open MPI. I'm installing:
> - deal.ii-8.1.0
> - Trilinos 11.4.3
> - PETSc 3.4.3*
> - MUMPS 4.10.0
> - p4est 0.3.4.2

That is a weird setup. Why do you want Trillinos, PETSc, and MUMPS?
I can understand you may want to do calculations in multiple different
ways - but is this really necessary?

> *I know you say to use <=3.4.2 but I was having issue installing that
> version and having dealii recognizing it but it recognized 3.4.3 from YaST
> easily.

@ developers Do we say this? Where? This is one version patch ahead of
me (time is my big problem right now). Is there a problem with
petsc-3.4.3 we need to deal with? I can have a look on wednesday on
this.

I am not 100% sure what that means. What are these "tweaks"?
Have you made a "custom MUMPS" and now have a linker error?

Can you install PETSc with --download-mumps=1. Alot of work by various
went into getting the dealii::SparseDirectMUMPS working in deal.II -
and it is a good interface. There is an alternative interface to MUMPS
through PETScWrappers (he lives in petsc_solver.*).
dealii::PETScWrappers::SparseDirectMUMPS.

In my opinion, if you want PETSc *and* MUMPS, this is by far the
easiest way to get it working. Installing MUMPS is tricky. ;-)
Try it out with petsc-3.4.2 - It should definately work.

Hope that helps you get started.

Best,
Toby

[Timo Heister]

>> *I know you say to use <=3.4.2 but I was having issue installing that
>> version and having dealii recognizing it but it recognized 3.4.3 from YaST
>> easily.
>
> @ developers Do we say this? Where? This is one version patch ahead of
> me (time is my big problem right now). Is there a problem with
> petsc-3.4.3 we need to deal with? I can have a look on wednesday on
> this.

http://www.dealii.org/developer/external-libs/petsc.html:

"Note: The latest version of PETSc tested is 3.4.2. Major releases
after this version may cause problems, so we recommend sticking to
this version if at all possible."

--
Timo Heister
http://www.math.clemson.edu/~heister/

[Toby Young]

Acha, i see. Thanks Timo.
I will run tests on 3.4.3 on wednesday. I will update the notes if
things look ok.
That release somehow slipped through my net ;-)

> --
> The deal.II project is located at http://www.dealii.org/
> For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to dealii+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Chad Balen]

> That is a weird setup. Why do you want Trillinos, PETSc, and MUMPS?
> I can understand you may want to do calculations in multiple different
> ways - but is this really necessary?

I'm a new grad student so I'm pretty new to everything and the reason why I had to install all of it. If I'm not mistaken its because:

i) Mumps - due to us having to solve highly nonlinear problems so we need a direct solver

ii) Trilinos and Petsc - to get the program to run in parrarell, started off with just Trilinos then my supervisor wanted me to also get Petsc since some of the dealii examples used both.

> I am not 100% sure what that means. What are these "tweaks"?
> Have you made a "custom MUMPS" and now have a linker error?

If you looked below you could see that I was getting some issues when installing Mumps (such as the Scalapacks installation) and I mentioned how I fixed them based on other people having similar issues.

So yeah, I made a custom mumps, it finally compiled and dealii recognized it. But when I try to use sparce_direct.h it tells me:

undefined reference to `dealii::SparseDirectMUMPS::SparseDirectMUMPS()'

which is why I figured there is a linking issue.

> Can you install PETSc with --download-mumps=1.

The problem with Petsc is that when I installed Petsc by hand (v3.4.2) and then tried to point dealii to its directory it never recognized my Petsc installation, even though no errors where thrown during the Petsc installation. However, it recognized the Petsc installation that YaST did. So I just hoped that any changes in 3.4.3 would be insignificant and gave up on trying to get v3.4.2 to work.

So short answer yes I can compile Petsc but dealii won't recognize it when I give it the Petsc directory.

Also, they actually have version 3.4.4 out as well on their website.

> There is an alternative interface to MUMPS through PETScWrappers (he > lives in petsc_solver.*). dealii::PETScWrappers::SparseDirectMUMPS

Ok, so I changed the head file to: petsc_solver.h and I changed
SparseDirectMUMPS  solver; TO
PETScWrappers::SparseDirectMUMPS  solver;

however it was giving me the following error:

Scanning dependencies of target step-3

[ 50%] Building CXX object CMakeFiles/step-3.dir/step-3.cc.o
/home/cbalen/Installed_Programs/deal.II/Install/examples/step-3/Mumps/step-3.cc: In member function ‘void Step3::solve()’:
/home/cbalen/Installed_Programs/deal.II/Install/examples/step-3/Mumps/step-3.cc:503:3: error: ‘PETScWrappers’ has not been declared
/home/cbalen/Installed_Programs/deal.II/Install/examples/step-3/Mumps/step-3.cc:504:3: error: ‘solver’ was not declared in this scope
make[3]: *** [CMakeFiles/step-3.dir/step-3.cc.o] Error 1
make[2]: *** [CMakeFiles/step-3.dir/all] Error 2
make[1]: *** [CMakeFiles/run.dir/rule] Error 2
make: *** [run] Error 2

So to recap before the function was unreferenced now its undefined. I went to the petsc_solver.h file and commented out #ifdef DEAL_II_WITH_PETSC because it doesn't appear to be compiled at compile time. Then it told me:

/home/cbalen/Installed_Programs/deal.II/Install/include/deal.II/lac/petsc_solver.h:29:24: fatal error: petscksp.h: No such file or directory

So I put in the full path name to petscksp.h hoping that it would work but it seems that the header file within petsc_solver.h which was petscksp.h could also not be found.

Side Note: I also had to put in the full path name to sparse_direct.h and petsc_solver.h, maybe suggesting there's some issue with dealii finding the header files.

Continuing on if I change it to the full path then it gives me a new header that it can't find even if it's in the same folder like what happens for petscksp.h and petscpc.h. So there's some issue with the files finding the header files.

--

Chad Balen

Energy Systems Design Laboratory,

3-23 Mechanical Engineering Building,

University of Alberta,
Edmonton, AB

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---

You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/7uZgTt-YZBw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.

[Bruno Turcksin]

Chad,

So to recap before the function was unreferenced now its undefined. I went to the petsc_solver.h file and commented out #ifdef DEAL_II_WITH_PETSC because it doesn't appear to be compiled at compile time. Then it told me:

Don't do that, you will just make it harder for us to understand what's going on.
 

/home/cbalen/Installed_Programs/deal.II/Install/include/deal.II/lac/petsc_solver.h:29:24: fatal error: petscksp.h: No such file or directory

So I put in the full path name to petscksp.h hoping that it would work but it seems that the header file within petsc_solver.h which was petscksp.h could also not be found.

Side Note: I also had to put in the full path name to sparse_direct.h and petsc_solver.h, maybe suggesting there's some issue with dealii finding the header files.

Continuing on if I change it to the full path then it gives me a new header that it can't find even if it's in the same folder like what happens for petscksp.h and petscpc.h. So there's some issue with the files finding the header files.

You don't need put the full path. Are the steps using PETSc and Trilinos working ? When I get some weird errors like that I usually blame the CMake cache, can you delete your build directory and your install directory and try to recompile deal.ii.

Best,

Bruno

[Chad Balen]

Hey Guys,

You don't need put the full path.

Yeah, I guess that was the first sign I was going to have issues with using step-3 with mumps when I had to put the full path to sparse_direct.h

Are the steps using PETSc and Trilinos working ?

Yeah that was the weird thing all all the steps I tried were compiling and working, except step-19 because there's no make run command.

When I get some weird errors like that I usually blame the CMake cache, can you delete your build directory and your install directory and try to recompile deal.ii.

So I went back and recompiled and I got a similar issue during compiling that I thought I had fixed. If you look in particular at my deal_Standard_make (All files are attached in deal_configure_attempt2.tar.gz).

Note: deal_Error_make says gives make: *** [all] Error 2, at the end of its file

but what's happening is that during make it quits making anywhere between 60% - 70%. I thought it might have had to do with Mumps when I realized I had Blacs, Scalapack, and Mumps pointing to different MPI directories (one of them was pointing to mpich instead of openmpi) in their make files. I fixed that and it compiled fine, but I guess it was just luck then. Could it possibly be my computer that's the problem? It does take 2 hours to compile dealii with -j4 and is frozen for the most part of that 2 hours.

Another interesting note is that all the tests still compiled and didn't fail.

--

Chad Balen

Energy Systems Design Laboratory,

3-23 Mechanical Engineering Building,

University of Alberta,
Edmonton, AB

--

[Bruno Turcksin]

Chad,

>
>
> So I went back and recompiled and I got a similar issue during
> compiling that I thought I had fixed. If you look in particular at my
> deal_Standard_make (All files are attached in
> deal_configure_attempt2.tar.gz).
>
> Note: deal_Error_make says gives make: *** [all] Error 2, at the end
> of its file
>

In your file deal_Error_make, you have this error line 1522:

c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugs.opensuse.org/> for instructions.
make[2]: ***
[source/meshworker/CMakeFiles/obj_meshworker.debug.dir/mesh_worker_vector_selector.cc.o] Error 4
make[1]: *** [source/meshworker/CMakeFiles/obj_meshworker.debug.dir/all]
Error 2
make[1]: *** Waiting for unfinished jobs....

Which compiler are you using ? If you are using gcc, you can type gcc
--version to get the version of gcc that you are using.

> but what's happening is that during make it quits making anywhere
> between 60% - 70%. I thought it might have had to do with Mumps when I
> realized I had Blacs, Scalapack, and Mumps pointing to different MPI
> directories (one of them was pointing to mpich instead of openmpi) in
> their make files. I fixed that and it compiled fine, but I guess it
> was just luck then. Could it possibly be my computer that's the
> problem? It does take 2 hours to compile dealii with -j4 and is frozen
> for the most part of that 2 hours.

Yes, that is very slow. It doesn't take that long on my laptop.

Best,

Bruno

[Chad Balen]

Hey Bruno,

Which compiler are you using ? If you are using gcc, you can type gcc
--version to get the version of gcc that you are using.

My output from terminal is:

$ gcc --version

gcc (SUSE Linux) 4.7.2 20130108 [gcc-4_7-branch revision 195012]
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

--

Chad Balen

Energy Systems Design Laboratory,

3-23 Mechanical Engineering Building,

University of Alberta,
Edmonton, AB

Best,

Bruno

[Bruno Turcksin]

Well, I don't know. It should work but since it is an internal error of
your compiler I would try an newer version of the compiler (4.7.3 or
4.8.1) but you have to make sure that openmpi is using the new compiler.

Bruno

[Wolfgang Bangerth]

On 03/17/2014 03:27 PM, Toby Young wrote:
> Acha, i see. Thanks Timo.
> I will run tests on 3.4.3 on wednesday. I will update the notes if
> things look ok.

Thanks, Toby,

I think we should take the note out. It is impossible to keep these things up
to date and we will hear if there is a newer version that is incompatible.

Best
W.


--
------------------------------------------------------------------------
Wolfgang Bangerth email: bang...@math.tamu.edu
www: http://www.math.tamu.edu/~bangerth/

[Wolfgang Bangerth]

> Note: deal_Error_make says gives make: *** [all] Error 2, at the end of its file
>
> but what's happening is that during make it quits making anywhere between 60%
> - 70%. I thought it might have had to do with Mumps when I realized I had
> Blacs, Scalapack, and Mumps pointing to different MPI directories (one of them
> was pointing to mpich instead of openmpi) in their make files. I fixed that
> and it compiled fine, but I guess it was just luck then. Could it possibly be
> my computer that's the problem? It does take 2 hours to compile dealii with
> -j4 and is frozen for the most part of that 2 hours.

I think you are running out of memory. (How much do you have?) That explains
why the compiler processes are terminated, and it explains why your machine
feels frozen. Try running with only -j1.

[Toby D. Young]

> > I will run tests on 3.4.3 on wednesday. I will update the notes if
> > things look ok.
>

> I think we should take the note out. It is impossible to keep these
> things up to date and we will hear if there is a newer version that
> is incompatible.

@Wolfgang I agree, I've updated the notes in that way.

@ALL Additionally, to confirm on this thread, deal.II compiles, links,
and runs perfectly well with:

petsc-3.4.3, petsc-3.4.4,
and
slepc-3.4.3 (there is no slepc-3.4.4 yet - its still in the pipes).

There appear to be no compatibility problems & I encourage users of
PETSc (such as myself) to upgrade whenever possible.

Best,
Toby

[Timo Heister]

> I think we should take the note out. It is impossible to keep these things up to date and we will hear if there is a newer version that is incompatible.

I disagree. There is no harm in using a PETSc version that is a couple
of months old. It's not that things fundamentally change from revision
to revision (and if things change, stuff likely breaks). Remember the
trouble we had with new Trilinos releases that compiled fine, but
caused hard to find bugs? This causes a lot of frustration especially
for less experienced users.
Even if a new release doesn't compile, it is still very frustrating
having to guess which version is safe to use. And realizing that this
is the problem in the first place and not some compiler/OS issues.
Remember that not everyone will write on the mailing list if their
configuration doesn't work.

I would prefer we keep the notice in and put a note in the release
task list to update the PETSc and Trilinos version numbers before a
release (and run the testsuite with the new version first!).

[Wolfgang Bangerth]

On 03/19/2014 10:35 AM, Timo Heister wrote:
>> I think we should take the note out. It is impossible to keep these things up to date and we will hear if there is a newer version that is incompatible.
>
> I disagree.

Didn't you just start your previous message with

> @Wolfgang I agree, I've updated the notes in that way.

? :-)

> There is no harm in using a PETSc version that is a couple
> of months old. It's not that things fundamentally change from revision
> to revision (and if things change, stuff likely breaks). Remember the
> trouble we had with new Trilinos releases that compiled fine, but
> caused hard to find bugs? This causes a lot of frustration especially
> for less experienced users.
> Even if a new release doesn't compile, it is still very frustrating
> having to guess which version is safe to use. And realizing that this
> is the problem in the first place and not some compiler/OS issues.
> Remember that not everyone will write on the mailing list if their
> configuration doesn't work.
>
> I would prefer we keep the notice in and put a note in the release
> task list to update the PETSc and Trilinos version numbers before a
> release (and run the testsuite with the new version first!).

That works, too. But then can we rewrite the message to just say that
"we tested up to version x.y and recommend these in case you encounter
problems with newer versions", rather than "...Major releases
after this version may cause problems"? I find the wording of the
current text too strong.

[Timo Heister]

> Didn't you just start your previous message with
>
>> @Wolfgang I agree, I've updated the notes in that way.

Toby != Timo.

:-)

> That works, too. But then can we rewrite the message to just say that "we
> tested up to version x.y and recommend these in case you encounter problems
> with newer versions", rather than "...Major releases
> after this version may cause problems"? I find the wording of the current
> text too strong.

Sure. @Toby, can you please add the blurb back in with Wolfgang's suggestion?

[Toby Young]

>> Didn't you just start your previous message with
>>
>>> @Wolfgang I agree, I've updated the notes in that way.
>
> Toby != Timo.

*That* was the funniest communication confusion I've seen on this
mailing list. :-D

> Sure. @Toby, can you please add the blurb back in with Wolfgang's suggestion?

Yep, okej, it is done. I prefer blowing away that kind of advice,
though I understand there is a merit to it being there. I will take a
another look at the petsc.html and slepc.html docs later when
slepc-3.4.4 is out. I am working on LAPACKFullMatrix at the moment, so
I have a slightly different mind set right now....

Best
Toby

[Chad Balen]

Hey Guys,

So after some work I have some updates

I think you are running out of memory. (How much do you have?) That explains why the compiler processes are terminated, and it explains why your machine feels frozen. Try running with only -j1.

Running on -j1 surprising allowed it to compile faster and I no longer got that compiler error. So it looks like it was a memory issue.

Are the steps using PETSc and Trilinos working ?

Since I was able to get these steps to work it made me wonder. So I went back and compared step 3 to step 17 to see of any major differences when using the PETSc wrappers.

I noticed that there are some subtle differences throughout the code that I didn't bother to fix in step-3, for instance when initiating the solution vector

- step-3: Vector<double>       solution;

- step-17: PETScWrappers::MPI::SparseMatrix system_matrix;

Instead of trying to fix step-3 I just decided to try and switch step-17 to use MUMPS. So I altered the solve() function so it contains:

SolverControl           solver_control (solution.size(),
                                            1e-8*system_rhs.l2_norm());
PETScWrappers::SparseDirectMUMPS solver (solver_control,
                                                                               mpi_communicator);
solver.solve (system_matrix, solution, system_rhs);

// I get the same error if I comment out the next 3 lines

PETScWrappers::Vector localized_solution (solution);
hanging_node_constraints.distribute (localized_solution);
solution = localized_solution;
return solver_control.last_step();

And I now get the following error when I make run:

Scanning dependencies of target step-17
[ 50%] Building CXX object CMakeFiles/step-17.dir/step-17.cc.o
In file included from /usr/lib64/mpi/gcc/openmpi/include/openmpi/ompi/mpi/cxx/mpicxx.h:204:0,
                 from /usr/lib64/mpi/gcc/openmpi/include/mpi.h:2087,
                 from /home/cbalen/Install_Dealii/deal.II/Install/include/deal.II/base/mpi.h:24,
                 from /home/cbalen/Install_Dealii/deal.II/Install/include/deal.II/base/utilities.h:22,
                 from /home/cbalen/Install_Dealii/deal.II/Install/include/deal.II/base/tensor.h:23,
                 from /home/cbalen/Install_Dealii/deal.II/Install/include/deal.II/base/function.h:25,
                 from /home/cbalen/Install_Dealii/deal.II/Install/examples/step-17/step-17.cc:25:
/usr/lib64/mpi/gcc/openmpi/include/openmpi/ompi/mpi/cxx/constants.h:54:50: warning: extra ‘;’ [-pedantic]
Linking CXX executable step-17
[ 50%] Built target step-17
[100%] Run step-17 with Debug configuration
Cycle 0:
   Number of active cells:       64
   Number of degrees of freedom: 162 (by partition: 162)

--------------------------------------------------------
An error occurred in line <806> of file </home/cbalen/Install_Dealii/deal.II/src/source/lac/petsc_solver.cc> in function
    void dealii::PETScWrappers::SparseDirectMUMPS::solve(const dealii::PETScWrappers::MatrixBase&, dealii::PETScWrappers::VectorBase&, const dealii::PETScWrappers::VectorBase&)
The violated condition was:
    false
The name and call sequence of the exception was:
    ExcMessage ("Your PETSc installation does not include a copy of " "MUMPS package necessary for this solver")
Additional Information:
Your PETSc installation does not include a copy of MUMPS package necessary for this solver

Stacktrace:
-----------
#0  /home/cbalen/Install_Dealii/deal.II/Install/lib/libdeal_II.g.so.8.1.0: dealii::PETScWrappers::SparseDirectMUMPS::solve(dealii::PETScWrappers::MatrixBase const&, dealii::PETScWrappers::VectorBase&, dealii::PETScWrappers::VectorBase const&)
#1  ./step-17: Step17::ElasticProblem<2>::solve()
#2  ./step-17: Step17::ElasticProblem<2>::run()
#3  ./step-17: main
--------------------------------------------------------

[secanell-grad06:13000] *** Process received signal ***
[secanell-grad06:13000] Signal: Aborted (6)
[secanell-grad06:13000] Signal code:  (-6)
[secanell-grad06:13000] [ 0] /lib64/libpthread.so.0(+0xf1f0) [0x7f34b03f91f0]
[secanell-grad06:13000] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x7f34af9bc3d5]
[secanell-grad06:13000] [ 2] /lib64/libc.so.6(abort+0x148) [0x7f34af9bd858]
[secanell-grad06:13000] [ 3] /home/cbalen/Install_Dealii/deal.II/Install/lib/libdeal_II.g.so.8.1.0(_ZN6dealii18deal_II_exceptions9internals5abortERKNS_13ExceptionBaseEb+0x62) [0x7f34ba962f97]
[secanell-grad06:13000] [ 4] ./step-17(_ZN6dealii18deal_II_exceptions9internals11issue_errorINS_18StandardExceptions10ExcMessageEEEvNS1_17ExceptionHandlingEPKciS7_S7_S7_T_+0x6a) [0x43088c]
[secanell-grad06:13000] [ 5] /home/cbalen/Install_Dealii/deal.II/Install/lib/libdeal_II.g.so.8.1.0(_ZN6dealii13PETScWrappers17SparseDirectMUMPS5solveERKNS0_10MatrixBaseERNS0_10VectorBaseERKS5_+0x97) [0x7f34ba3f2b61]
[secanell-grad06:13000] [ 6] ./step-17(_ZN6Step1714ElasticProblemILi2EE5solveEv+0xfd) [0x434533]
[secanell-grad06:13000] [ 7] ./step-17(_ZN6Step1714ElasticProblemILi2EE3runEv+0x1f2) [0x4321e4]
[secanell-grad06:13000] [ 8] ./step-17(main+0x6c) [0x42b075]
[secanell-grad06:13000] [ 9] /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f34af9a8a15]
[secanell-grad06:13000] [10] ./step-17() [0x42aea9]
[secanell-grad06:13000] *** End of error message ***
make[3]: *** [CMakeFiles/run] Aborted
make[2]: *** [CMakeFiles/run.dir/all] Error 2

make[1]: *** [CMakeFiles/run.dir/rule] Error 2
make: *** [run] Error 2

So I guess my YaST version of PETSc (petsc-openmpi-devel v3.4.3) was not configured with MUMPS. So I went and tried to get my version of PETSc to configure properly. However, when I try to compile PETSc with MUMPS it never recognizes my SCALAPACK installation (more on this at the end). So if I configure PETSc to just use it's own downloaded version of SCALAPACK and MUMPS (As Toby Young suggested):

My PETSc configure:

./config/configure.py --with-shared-libraries=1 --with-x=0 --with-mpi-dir=/usr/lib64/mpi/gcc/openmpi --download-hypre=1 --download-scalapack=1 --download-mumps=1 --download-ptscotch=1 2>&1 | tee ../1_PETSc_configure_report.txt

PETSc compiles fine but when I configure dealii I get this error:

CMake Error at cmake/configure/configure_3_petsc.cmake:133 (MESSAGE):
 

  Could not find the petsc library!

 

  Please ensure that the petsc library version 3.0.0 or newer is installed on
  your computer and is configured with the same mpi options as deal.II

  If the library is not at a default location, either provide some hints

  for the autodetection:

  PETSc installed with --prefix=<...> to a destination:

      $ PETSC_DIR="..." cmake <...>
      $ cmake -DPETSC_DIR="..." <...>

  PETSc compiled in source tree:

      $ PETSC_DIR="..."  PETSC_ARCH="..." cmake <...>
      $ cmake -DPETSC_DIR="..." -DPETSC_ARCH="..." <...>

  or set the relevant variables by hand in ccmake.

 

Call Stack (most recent call first):
  /home/cbalen/Install_Dealii/deal.II/Build2/CMakeFiles/CMakeTmp/macro_configure_feature.tmp:1 (FEATURE_PETSC_ERROR_MESSAGE)
  cmake/macros/macro_configure_feature.cmake:72 (INCLUDE)
  cmake/macros/macro_configure_feature.cmake:255 (RUN_COMMAND)
  cmake/configure/configure_3_petsc.cmake:152 (CONFIGURE_FEATURE)
  CMakeLists.txt:119 (INCLUDE)


-- Configuring incomplete, errors occurred!
make: *** No targets specified and no makefile found.  Stop.
make: *** No rule to make target `install'.  Stop.
make: *** No rule to make target `test'.  Stop.

If I remove --download-scalapack=1 --download-mumps=1 --download-ptscotch=1 as I had it when I tried to install it by hand I get the same error. As well I get the same error if I use --with-mpi=1. So I'm left to conclude that something is wrong with how I'm configuring PETSc but I'm following dealii instructions. Any ideas? Attached is the configure report from dealii (1_Deal_configure_report_MY_PETSC.txt).

Possible future PETSc issue I'm going to have?

If I do get dealii to recognize PETSc am I wrong in this being a future issue I'm going to have:
- PETSc needs SCALAPACK >= 2.0 since these versions contain BLACS within them,
- BUT dealii uses SCALAPACK <= 1.8 because dealii hasn't been setup to find BLACS within SCALAPACK

So I'm going to probably need PETSc v3.1.0 in the future because that release came out before SCALAPACK v2.0

--

Chad Balen

Energy Systems Design Laboratory,

3-23 Mechanical Engineering Building,

University of Alberta,
Edmonton, AB

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---

You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/7uZgTt-YZBw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.

[Bruno Turcksin]

Chad,

when you install PETSc through Yast, PETSc is installed in a directory
that deal.II will check. However, when you install PETSc yourself,
deal.II does not know where to look for. For this reason, you have to
give the directory using cmake -DPETSC_DIR=/path/to/petsc
-DPETSC_ARCH=architecture If it is what you have done check that there
is no typo. The error that you see is not due to the way you have
configured PETSc. You have an error because deal.II is looking in the
wrong directory.

Best,

Bruno

[Chad Balen]

Hey Bruno,

My normal cmake script for dealii is:

cmake \
  ../src \
  -DCMAKE_INSTALL_PREFIX=../Install2 \
  -DDEAL_II_WITH_MPI=ON \
  -DDEAL_II_WITH_MUMPS=ON \
  -DSCALAPACK_DIR=../../SCALAPACK/scalapack-1.8.0 \
  -DBLACS_DIR=../../BLACS/BLACS \
  -DMUMPS_DIR=../../MUMPS/MUMPS_4.10.0 \
  -DDEAL_II_WITH_TRILINOS=ON \
  -DTRILINOS_DIR=../../Trilinos/Install \
  -DDEAL_II_WITH_P4EST=ON \
  -DP4EST_DIR=../../p4est/Install \
  -DDEAL_II_WITH_PETSC=ON \
  -DPETSC_DIR=../../PETSc/petsc-3.4.2 \
  -DPETSC_ARCH=x86_64 \
  2>&1 | tee ../1_Deal_configure_report.txt

I've also changed -DPETSC_DIR to

-DPETSC_DIR=/home/cbalen/Install_Dealii/PETSc/petsc-3.4.2 \

just now where I copied and pasted the path from dolphin so it's not a typing issue.

As well, if I point the MUMPS path to the external dependencies within PETSc it gives the same error as above but with MUMPS not being found instead of PETSc (I'm guessing because it looks for MUMPS first and PETSc configures MUMPS differently than dealii wants).

--

Chad Balen

Energy Systems Design Laboratory,

3-23 Mechanical Engineering Building,

University of Alberta,
Edmonton, AB

[Bruno Turcksin]

Chad,

are you sure that you are using the right architecture ? I haven't
compiled PETSc in a long time but for me the architecture was:
arch-linux2-c-debug even if I'm using a x86_64 processor.

Best,

Bruno

[Chad Balen]

Hey Bruno,

I changed my -DPETSC_ARCH=arch-linux2-c-debug and it configured properly. I wish that this was given more of a mention on the dealii PETSc installation page. Dealii is re-compiling now with my installation of petsc I'll let you guys know afterwards how this affects my steps-17 Mumps variant

--

Chad Balen

Energy Systems Design Laboratory,

3-23 Mechanical Engineering Building,

University of Alberta,
Edmonton, AB

[Wolfgang Bangerth]

Chad,

> I changed my -DPETSC_ARCH=arch-linux2-c-debug and it configured properly. I
> wish that this was given more of a mention on the dealii PETSc installation
> page.

doc/external-libs/petsc.html says this:

PETSc usually requires you to set the
environment variables <code>PETSC_DIR</code> and <code>PETSC_ARCH</code>
to a path to PETSc and denoting the architecture for which PETSc is
compiled (a string you can choose however you like, it is simply
intended to identify one of possibly several different PETSc
installations). If these environment variables are set, then
<acronym>deal.II</acronym> will pick them up during
configuration, and store them. It will then also recognize that
PETSc shall be used, and enable the wrapper classes.

I've modified this slightly to make it clear that the PETSC_ARCH you pass to
deal.II must be the same that you used when you compiled PETSc, but that it is
otherwise an arbitrary string you can choose however you want.

[Chad Balen]

Hey Guys,

So I got dealii installed re-ran step-17 mumps variant but it gave the same error as before when I used YaSt PETSc which says that my version of PETSc does not include MUMPS. Here's the Error Message:

An error occurred in line <806> of file </home/cbalen/Install_Dealii/deal.II/src/source/lac/petsc_solver.cc> in function
    void dealii::PETScWrappers::SparseDirectMUMPS::solve(const dealii::PETScWrappers::MatrixBase&, dealii::PETScWrappers::VectorBase&, const dealii::PETScWrappers::VectorBase&)
The violated condition was:
    false
The name and call sequence of the exception was:
    ExcMessage ("Your PETSc installation does not include a copy of " "MUMPS package necessary for this solver")
Additional Information:
Your PETSc installation does not include a copy of MUMPS package necessary for this solver

And my configure script again is:

./config/configure.py --with-shared-libraries=1 --with-x=0 --with-mpi=1 --download-hypre=1 --download-scalapack=1 --download-mumps=1 --download-ptscotch=1 2>&1 | tee ../1_PETSc_configure_report.txt

Shouldn't it be sufficient to tell PETSc to install MUMPS?

--

Chad Balen

Energy Systems Design Laboratory,

3-23 Mechanical Engineering Building,

University of Alberta,
Edmonton, AB

[Toby Young]

If I recall correctly, yes, that should be sufficient. Are you
absolutely sure that deal.II is picking up on that PETSc installation
and not another one you left lying around? Can you please check which
PETSc installation deal.II has found in your detailed.log. I can have
closer look at this tomorrow at school in nobody jumps in with a
solution. Best, Toby

BTW: If deal.II has picked the wrong PETSc installation, you can use
-DPETSC_DIR=path/to/petsc and -DPETSC_ARCH=arch to directly tell
deal.II to use your PETSc installation that has MUMPS.

> You received this message because you are subscribed to the Google Groups
> "deal.II User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an

[Chad Balen]

Hey Guys,

I always put the -DPETSC_DIR and I checked that I had set the correct path, I also checked the detailed.log and it said it was the correct path to the PETSc I was using. So, I decided I would just delete all the installed programs and just do a completely clean build of all software to use with deal.II and now PETSc (with its built in MUMPS) works. I don't know what was wrong but now everything works. Thanks everyone for all the help and quick replies it was much appreciated.

--

Chad Balen

Energy Systems Design Laboratory,

3-23 Mechanical Engineering Building,

University of Alberta,
Edmonton, AB