Trouble building parmetis with intel@15 (mpicc unable to compile a simple program)

[Kelly Thompson]

Hi,

Yesterday, I asked this question in the discussion for issue #761 (https://github.com/LLNL/spack/issues/761), but decided that this forum might be a better place to have the discussion. I am working from a clean develop source tree (minus a couple of user-level modifications mentioned below).

I want to provide parmetis%in...@15.0.5 via spack.  On our HPC machines the Intel compiler is established by environment variables  (I have replaced a valid but long system specific paths with the string 'somepath').  

module-whatis Intel C/C++/Fortran 15.0.5
setenv INTEL /somepath/intel/2015.5.223/composer_xe_2015.5.223
setenv CC /somepath/intel/2015.5.223/composer_xe_2015.5.223/bin/intel64/icc
setenv CXX /somepath/intel/2015.5.223/composer_xe_2015.5.223/bin/intel64/icpc
setenv F77 /somepath/intel/2015.5.223/composer_xe_2015.5.223/bin/intel64/ifort
setenv FC /somepath/intel/2015.5.223/composer_xe_2015.5.223/bin/intel64/ifort
setenv INTEL_LICENSE_FILE /somepath/intel/license.dat
prepend-path LD_LIBRARY_PATH /somepath/intel/2015.5.223/composer_xe_2015.5.223/compiler/lib/intel64
prepend-path PATH /somepath/intel/2015.5.223/composer_xe_2015.5.223/bin/intel64
prepend-path MANPATH /somepath/intel/2015.5.223/composer_xe_2015.5.223/man/en_US  

To allow the spack compile wrappers to work, I have generated an intel.cfg file with this content:

#intel.cfg

-Xlinker -rpath=/somepath/intel/2015.5.223/composer_xe_2015.5.223/compiler/lib/intel64

and I set the environment variables ICCCFG, ICPCCFG and IFORTCFG to point to this file.  These changes allows me to build metis%in...@15.0.5 and other tools. Before building parmetis, I also setup etc/spack/package.yaml to point to our custom OpenMPI installation:

# packages.yaml

packages:
  openmpi:
    paths:    
      openmpi@1.6.5%intel@15.0.5=linux-x86_64: /somepath/openmpi/1.6.5-intel-15.0.5
      openmpi@1.6.5%gcc@4.4.7=linux-x86_64: /somepath/openmpi/1.6.5-gcc-4.4.7

To build parmetis, I issued the command:

spack install parmetis%intel ^openmpi@1.6.5%intel@15.0.5

Based on the generated log files, the environment looks correct.  However, because parmetis sets CC=mpicc and CXX=mpicxx, cmake fails to find a compiler that works.  It appears that the compile wrappers are not respecting the environment variables  ICCCFG and ICPCCFG because the build error is:

-- Check for working C compiler: /somepath/openmpi/1.6.5-intel-15.0.5/bin/mpicc
-- Check for working C compiler: /somepath/openmpi/1.6.5-intel-15.0.5/bin/mpicc -- broken
CMake Error at /somepath/spack/opt/spack/linux-x86_64/gcc-4.4.7/cmake-3.5.1-k6ifgpngyer2b7j2sfmyfmtwgpn2uygr/share/cmake-3.5/Modules/CMakeTestCCompiler.cmake:61 (message):
  The C compiler
  "/somepath/openmpi/1.6.5-intel-15.0.5/bin/mpicc" is not able to compile a simple test program.

<removed extraneous information>
 
  /somepath/openmpi/1.6.5-intel-15.0.5/bin/mpicc:
  error while loading shared libraries: libimf.so: cannot open shared object
  file: No such file or directory

FWIW - I can build parmetis%gcc ^openmpi@1.6.5%gcc without any trouble and I can build parmetis with Intel (via openmpi's mpicc) by hand.

Does anyone know how to build parmetis with intel@15 with spack.  There appear to be a couple of ongoing activities related to this issue (#761, #638, #572), but I couldn't find any that work for me.

Thank you,

-kt

      

[luigi calori]

Hi Kelly, I've managed to install parmetis with:

spack install -v -j 20 parmetis%intel^ope...@1.6.5

I had a problem somehow similar to yours :

ld: cannot find -lxml2

I had it resolved adding dependency on libxml2 into
parmetis/package.py


     variant('gdb', default=False, description='Enables gdb support')
 
     depends_on('cmake @2.8:')  # build dependency
+    depends_on('libxml2') 
     depends_on('mpi')
 
     patch('enable_external_metis.patch')



I do not know reason for that, maybe is related  with openmpi as I get similar behaviour with one openmpi test

Let us know if it works in your case

[luigi calori]

Sorry, I think that the reason for requiring xml2 were caused by me using a non-standard cmake version ( i had forced spack to use cmake generated with gcc ).
This version of openmpi on my system build without problems

[Markus Frings]

Hi Kelly,

did you solve the issue? I have the same issue with using a precompiled openmpi.

Thanks,

Markus

[Kelly Thompson]

Markus,

There are two possible solutions to this issue depending on your system configuration.

1. If you normally load a module to setup the intel compiler, then you simply modify the compilers.yaml to mimic those load commands.  This is what I currently do. For example:

compilers:

- compiler:

    environment: {}

    extra_rpaths: []

    flags: {}

    modules:

      - intel/16.0.3

    operating_system: rhel6

    paths:

      cc: icc

      cxx: icpc

      f77: ifort

      fc: ifort

    spec: in...@16.0.3

    target: x86_64

The 'intel/16.0.3' module imports all of the necessary environment for spack.

2. Without a modulefile, you must setup Intel compiler environment variables manually by modifying the 'extra_rpaths' and/or 'environment' portions of your compilers.yaml. It has been a while since I have used this second method -- but I think you only need to set LD_LIBRARY_PATH in the 'extra_rpaths' entry.  You might need to test a few setups to determine what works for your system.

-kt
-----
Kelly (KT) Thompson
k...@eklaj.org

--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+unsubscribe@googlegroups.com.
To post to this group, send email to sp...@googlegroups.com.
Visit this group at https://groups.google.com/group/spack.
For more options, visit https://groups.google.com/d/optout.