Compilation deal.II 9.5

[giuseppe orlando]

Hello everyone,

I'm writing this post because I'm having some issues to compile the latest release. I would need a version which supports CUDA and MPI being CUDA-aware. Hence, I installed Kokkos using the flags -DKokkos_ENABLE_CUDA=ON and -DKokkos_ENABLE_CUDA_LAMBDA=ON. Then, I tried to build the Makefile specifying as CXX compiler the 'nvcc_wrapper' in Kokkos, namely using -DCMAKE_CXX_COMPILER=/path/kokkos/bin/nvcc_wrapper and using the flags -DDEAL_II_WITH_MPI=ON, -DDEAL_II_WITH_CUDA=ON, -DDEAL_II_MPI_WITH_DEVICE_SUPPORT=ON, -DKOKKOS_DIR=${Kokkos_dir}, but I get an error in finding MPI

 

I also tried to specify the compiler I have always used with the previous version via -DMPI_CXX_COMPILER flag, but I get the same error. Did someone encounter the same issue? Am I supposed to use 'nvcc_wrapper' also for MPI?

Thanks in advance.

Best,

Giuseppe

[Bruno Turcksin]

Giuseppe,

Your configuration line looks good. I use the same but I also export OMPI_CXX=/path/kokkos/bin/nvcc_wrapper

Best,

Bruno

[giuseppe orlando]

Dear Bruno,

thanks for the suggestion. I tried to follow it, but I still the get same error during the creation of the Makefile. Just to sum up I typed

"export OMPI_CXX=/project/sm77/bin/kokkos/bin/nvcc_wrapper"

"cmake -DCMAKE_INSTALL_PREFIX=/project/sm77/bin/dealII/9.5.1/ -DCMAKE_CXX_COMPILER=/project/sm77/bin/kokkos/bin/nvcc_wrapper -DDEAL_II_WITH_P4EST=ON -DP4EST_DIR=/project/sm77/bin/p4est/ -DDEAL_II_WITH_MPI=ON -DDEAL_II_WITH_CUDA=ON -DDEAL_II_MPI_WITH_DEVICE_SUPPORT=ON -DKOKKOS_DIR=/project/sm77/bin/kokkos ../."

At this point, I'm wondering if the problem could be related to how I installed Kokkos. This is the command to generate the Makefile for Kokkos and the installation went good, but at this point, I really don't know. 

"cmake -DCMAKE_CXX_COMPILER=/project/sm77/kokkos/bin/nvcc_wrapper -DCMAKE_INSTALL_PREFIX=/project/sm77/bin/kokkos -DCMAKE_CXX_FLAGS=-arch=sm_60 -DKokkos_ENABLE_CUDA_LAMBDA=ON -DKokkos_ENABLE_CUDA=ON -DKokkos_ARCH_PASCAL60=ON ../." 

Best,

Giuseppe

[Bruno Turcksin]

Giuseppe,

I think the issue is because you set -DCMAKE_CXX_FLAGS=-arch=sm_60 when compiling Kokkos. This is not necessary because Kokkos takes care of that when you use -DKokkos_ARCH_PASCAL60=ON. Since you set that as a CXX flags it might be passed to MPI when it shouldn't. The rest looks good.

Best,

Bruno

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/7VthT7AeI4Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/d4717449-8c4a-48d2-b0eb-c56a4381f631n%40googlegroups.com.

[giuseppe orlando]

Dear Bruno,

I tried to reinstall Kokkos removing the -DCMAKE_CXX_FLAGS option, but I get the same error during the creation of the Makefile for deal.II. I will try to figure out which might be the problem in the follwing days.

Best,

Giuseppe 

[Bruno Turcksin]

Giuseppe,

CMake should create an error log. Can you post it?

Best,

Bruno

To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/eb8caa99-28b8-455b-ad67-eac3e348931en%40googlegroups.com.

[giuseppe orlando]

Here, you can find both 'CMakeError.log' and 'CMakeOutput.log'

Best,

Giuseppe

[Bruno Turcksin]

What's strange is that when deal.II tries to compile an MPI code, it is using `nvcc_wrapper` as the compiler instead of `mpicxx`. Can you try to set `-DMPI_CXX_COMPILER=mpicxx`?

Bruno

To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/5a769fd1-3eb3-46a2-8f3c-210d08f39383n%40googlegroups.com.

[giuseppe orlando]

I agree. I had tried to set -DMPI_CXX_COMPILER flag, but it seems to be ineffective. It keeps using nvcc_wrapper for MPI.

Best

[Bruno Turcksin]

A few questions: are you configuring inside a clean directory? What's the output of mpicxx? Which version of cmake are you using?

To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/8d8473b5-6482-4830-9577-610df239c2ecn%40googlegroups.com.

[Giuseppe Orlando]

Yes, I’m inside a clean directory. The output of mpicxx is “/opt/cray/pe/craype/2.7.10/bin/CC”, which is the same that I used to install all the other versions. The cmake version is 3.17.0.

 

Best

 

Giuseppe Orlando

 

Da: Bruno Turcksin
Inviato: giovedì 20 luglio 2023 17:47
A: dea...@googlegroups.com
Oggetto: Re: [deal.II] Re: Compilation deal.II 9.5

 

To unsubscribe from this group and all its topics, send an email to dealii+un..@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/5a769fd1-3eb3-46a2-8f3c-210d08f39383n%40googlegroups.com.

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/7VthT7AeI4Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/8d8473b5-6482-4830-9577-610df239c2ecn%40googlegroups.com.

--

The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/7VthT7AeI4Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/CAGVt9ePTHoDMHMH7D%2BpOhWXL5Ms5-0ez6admDJZSqtFGfc8TFw%40mail.gmail.com.

 

[Bruno Turcksin]

You are using Cray that explains a lot. Cray does its own thing with MPI. If it's possible, you can use a "regular" MPI compiler instead of CC. Otherwise, you should be able to compile Kokkos using the cray compiler directly, i.e., you don't use `nvcc_wrapper`.

Bruno

To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/1A4FE272-C254-48D8-B596-94300A376B06%40hxcore.ol.