An issue in compiling on a cluster

[Alberto Salvadori]

Hi, 

your help in this issue is greatly appreciated.

I installed deal.II on a cluster using the spack distribution, very easily. I also run some examples provided with deal.II with no issues. Great job.

I am now using a library for tensor calculus, that apparently conflicts with the gcc compiler installed with spack. Specifically, the spack configuration uses /gcc-4.9.2 

opt/spack/linux-rhel7-x86_64/gcc-4.9.2/openmpi-2.0.2-kupebz5mdlwlfvbkcit3y56cg6r4bxg5/bin/mpic++

while this library needs gcc-6.2.0 or higher. The cluster is equipped with  mpich/3.2-gcc-6.2.0, but I am not sufficiently skilled to edit the configuration or the make files to solve the problem.

Any help?

Thanks

Alberto

[Bruno Turcksin]

Alberto,

you don't want spack to install mpi on a cluster take a look here https://spack.readthedocs.io/en/latest/getting_started.html#system-packages Also gcc 4.9 and gcc 6.2 are not compatible, so you need everything to be compiled by the same compiler, i.e. gcc 6.2.

Best,

Bruno

[Alberto Salvadori]

Thanks, Bruno.

If I understand well then, I have to add the  mpich/3.2-gcc-6.2.0 compiler to spack - I did this just now.

I checked that the file compilers.yaml includes the compiler and its path.The same file still includes the compiler gcc-4.9.2

Shall I delete the corresponding section  in the compilers.yaml file? 

Can I just run spack install dealii instead? 

Thanks!

Alberto

Alberto Salvadori
 Dipartimento di Ingegneria Civile, Architettura, Territorio, Ambiente e di Matematica (DICATAM)
 Universita` di Brescia, via Branze 43, 25123 Brescia
 Italy
 tel 030 3711239
 fax 030 3711312

e-mail:
 alberto....@unibs.it
web-pages:
 http://m4lab.unibs.it/faculty.html
 http://dicata.ing.unibs.it/salvadori

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Informativa sulla Privacy: http://www.unibs.it/node/8155

[Bruno Turcksin]

Alberto,

2017-04-04 16:04 GMT-04:00 Alberto Salvadori <alberto....@unibs.it>:
> If I understand well then, I have to add the mpich/3.2-gcc-6.2.0 compiler
> to spack - I did this just now.
> I checked that the file compilers.yaml includes the compiler and its
> path.The same file still includes the compiler gcc-4.9.2

That's good but you also want to make sure that spack will not compile
its own mpi. So you need to make sure that building mpich is set to
false.

> Shall I delete the corresponding section in the compilers.yaml file?

No you don't need to

> Can I just run spack install dealii instead?

You need to run something like spack install@dealii ^mpich %gcc@6.2
You need to make sure that spack will compile using mpich and that it
won't install it.

Best,

Bruno

[Alberto Salvadori]

Sorry, I am a bit confused.

"you need to make sure that building mpich is set to false." 

 - where I can set "building mpich" to false? Shall I edit a file  ~/.spack/packages.yaml that BTW currently I do not have? 

 - if so, would this be OK?

spack providers mpi

    intel-parallel-studio@cluster:+mpi  mpich@1:  mpich@3:  mvapich2@1.9  mvapich2@2.0:  ope...@1.6.5  ope...@1.7.5:  ope...@2.0.0:  spectrum-mpi

openmpi:
    version: [2.0.0]
    paths:
      ope...@2.0.0%g...@6.2.0: ...SOMEPATH.../m/mpich/3.2/gcc/6.2.0/bin ( the latter is the directory where mpic++ executable is )
    buildable: False

"You need to run something like spack  install@dealii ^mpich %gcc@6.2"

- sorry, I cannot understand this. I know I am bothering, but can you please explain this a bit further?

Thanks

Alberto

Alberto Salvadori
 Dipartimento di Ingegneria Civile, Architettura, Territorio, Ambiente e di Matematica (DICATAM)
 Universita` di Brescia, via Branze 43, 25123 Brescia
 Italy
 tel 030 3711239
 fax 030 3711312

e-mail:
 alberto....@unibs.it
web-pages:
 http://m4lab.unibs.it/faculty.html
 http://dicata.ing.unibs.it/salvadori

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bruno Turcksin]

2017-04-04 16:58 GMT-04:00 Alberto Salvadori <alberto....@unibs.it>:
> Sorry, I am a bit confused.
>
> "you need to make sure that building mpich is set to false."
> - where I can set "building mpich" to false? Shall I edit a file
> ~/.spack/packages.yaml that BTW currently I do not have?

Yes, you need to create the file if it doesn't exist.

> - if so, would this be OK?
>
> spack providers mpi
>
> intel-parallel-studio@cluster:+mpi mpich@1: mpich@3: mvapich2@1.9
> mvapich2@2.0: ope...@1.6.5 ope...@1.7.5: ope...@2.0.0: spectrum-mpi
>
>
> openmpi:
> version: [2.0.0]
> paths:
> ope...@2.0.0%g...@6.2.0: ...SOMEPATH.../m/mpich/3.2/gcc/6.2.0/bin (
> the latter is the directory where mpic++ executable is )
> buildable: False

This looks sketchy: you say openmpi but then the path says mpich...
Otherwise it looks good.

>
> "You need to run something like spack install dealii ^mpich %gcc@6.2"

> - sorry, I cannot understand this. I know I am bothering, but can you please
> explain this a bit further?

spack install dealii %gcc@6.2 -> install deal.II with the gcc 6.2
compiler. Because you have more than one compiler on your system, you
need to tell spack which compiler to use.
spack install dealii ^mpich -> I am not sure about this one. But by
default, spack uses openmpi but your path says that you are using
mpich. Thus, you need to force spack to use mpich instead of openmpi.
I am not sure this is the exact command though that's why I said it
should look like "spack install ^mpich %gcc@6.2"

Best,

Bruno

[Alberto Salvadori]

Hi Bruno,

I really appreciate your help. 

I got the right location of my openmpi and just mean to double check before running spack. I shall therefore:

1 - create a file  ~/.spack/linux/packages.yaml as follows:

openmpi:
    version: [2.0.1]
    paths:
      ope...@2.0.1%g...@6.2.0: ...SOMEPATH...openmpi/2.0.1/gcc/6.2.0/eth/bin 
    buildable: False

2 - spack install dealii %gcc@6.2

Correct? 

Thanks

Alberto

Alberto Salvadori
 Dipartimento di Ingegneria Civile, Architettura, Territorio, Ambiente e di Matematica (DICATAM)
 Universita` di Brescia, via Branze 43, 25123 Brescia
 Italy
 tel 030 3711239
 fax 030 3711312

e-mail:
 alberto....@unibs.it
web-pages:
 http://m4lab.unibs.it/faculty.html
 http://dicata.ing.unibs.it/salvadori

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bruno Turcksin]

Yeah it looks good.

Bruno

>> email to dealii+un...@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> Informativa sulla Privacy: http://www.unibs.it/node/8155
>

> --
> The deal.II project is located at http://www.dealii.org/
> For mailing list/forum options, see
> https://groups.google.com/d/forum/dealii?hl=en
> ---

> You received this message because you are subscribed to a topic in the

> Google Groups "deal.II User Group" group.

> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/dealii/95YGnSX7zHo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> dealii+un...@googlegroups.com.

[Alberto Salvadori]

Ouch ... something went wrong:

asalvad2 ~/.spack/linux $ cd $SPACK_ROOT

asalvad2 /scratch/asalvad2/spack $ spack install dealii %gcc@6.2

==> Error: <unknown line>: Additional properties are not allowed ('openmpi' was unexpected)

asalvad2 /scratch/asalvad2/spack $ module list

Currently Loaded Modulefiles:

  1) opt_local/1.0          2) gcc/6.2.0              3) ompi/2.0.1-gcc-6.2.0

asalvad2 /scratch/asalvad2/spack $ more ~/.spack/linux/packages.yaml 

openmpi:

    version: [2.0.1]

    paths:

      ope...@2.0.1%g...@6.2.0: /opt/crc/o/openmpi/2.0.1/gcc/6.2.0/eth/bin 

    buildable: False

Any further help?

Thanks

Alberto

Alberto Salvadori
 Dipartimento di Ingegneria Civile, Architettura, Territorio, Ambiente e di Matematica (DICATAM)
 Universita` di Brescia, via Branze 43, 25123 Brescia
 Italy
 tel 030 3711239
 fax 030 3711312

e-mail:
 alberto....@unibs.it
web-pages:
 http://m4lab.unibs.it/faculty.html
 http://dicata.ing.unibs.it/salvadori

[Alberto Salvadori]

Got it, sorry.

A

Alberto Salvadori
 Dipartimento di Ingegneria Civile, Architettura, Territorio, Ambiente e di Matematica (DICATAM)
 Universita` di Brescia, via Branze 43, 25123 Brescia
 Italy
 tel 030 3711239
 fax 030 3711312

e-mail:
 alberto....@unibs.it
web-pages:
 http://m4lab.unibs.it/faculty.html
 http://dicata.ing.unibs.it/salvadori

[Denis Davydov]

Bruno is right, you definitely don't want compile MPI yourself in a cluster. At the very top of the WiKi there is an example of how to configure Spack on a cluster with external MPI
https://github.com/dealii/dealii/wiki/deal.II-in-Spack
Probably you missed this.

I will edit the WiKi to stress this fact.

Regards,
Denis

[Denis Davydov]

HI Alberto,

On Tuesday, April 4, 2017 at 10:58:39 PM UTC+2, Alberto Salvadori wrote:

Sorry, I am a bit confused.

"you need to make sure that building mpich is set to false." 

 - where I can set "building mpich" to false? Shall I edit a file  ~/.spack/packages.yaml that BTW currently I do not have? 

 - if so, would this be OK?

spack providers mpi

    intel-parallel-studio@cluster:+mpi  mpich@1:  mpich@3:  mvapich2@1.9  mvapich2@2.0:  ope...@1.6.5  ope...@1.7.5:  ope...@2.0.0:  spectrum-mpi

openmpi:
    version: [2.0.0]
    paths:
      ope...@2.0.0%g...@6.2.0: ...SOMEPATH.../m/mpich/3.2/gcc/6.2.0/bin ( the latter is the directory where mpic++ executable is )
    buildable: False

it's almost correct, you need to provide a path to the prefix, i.e. without `bin`.

In other words, it's a path to the externally provided MPI package, not to the MPI compiler wrappers.

 

"You need to run something like spack  install@dealii ^mpich %gcc@6.2"

- sorry, I cannot understand this. I know I am bothering, but can you please explain this a bit further?

please have a look at this short presentation https://tgamblin.github.io/files/Gamblin-Spack-SC15-Talk.pdf

Essentially it's about adding extra constrains to what you want to build.

If you are a single user, those are generally easier to be added in the packages.yaml file as opposed to the command line.

Regards,

Denis.

[Denis Davydov]

p.s. it's a bit of a learning curve to set things up with Spack, but hopefully it does not take too much as there is a lot of documentation.

On the positive side, a lot of developers now give more and more attention to Spack and maintain their packages. I know that PETSc and Trilinos guys keep an eye on Spack.

Very recently, if I understand right, OpenFOAM developer revised its package in Spack and added more things. A developer of quantum mechanics code Abinit also contributed

a changes/fixes to their package. So in the ideal world you have a tool to build 1000+ packages where package developers are also maintaining some of those.

[Alberto Salvadori]

Thank you Denis and Bruno.

It worked out well, now, thanks to your suggestions. 

Deal.II community is just great.

To the next question!

Alberto

Alberto Salvadori
 Dipartimento di Ingegneria Civile, Architettura, Territorio, Ambiente e di Matematica (DICATAM)
 Universita` di Brescia, via Branze 43, 25123 Brescia
 Italy
 tel 030 3711239
 fax 030 3711312

e-mail:
 alberto....@unibs.it
web-pages:
 http://m4lab.unibs.it/faculty.html
 http://dicata.ing.unibs.it/salvadori

--

The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.