Linker error with Clang and Mac OSX

[Harsha H S]

Hi,

 I am trying to build Xyce Paralle with MPI support using the guide at

https://xyce.sandia.gov/documentation-tutorials/building-guide/#parallelOSXXyce

on my Mac system and I am encountering linker error with Thyra

  CXXLD    Xyce
ld: warning: dylib (/usr/local/Cellar/gcc/11.3.0_2/lib/gcc/11/libgfortran.dylib) was built for newer macOS version (12.4) than being linked (12.0)
ld: warning: dylib (/usr/local/Cellar/gcc/11.3.0_2/lib/gcc/11/libquadmath.dylib) was built for newer macOS version (12.4) than being linked (12.0)
ld: warning: object file (/usr/local/lib/libepetra.a(Epetra_dcrsmm.f.o)) was built for newer macOS version (12.4) than being linked (12.0)
ld: warning: object file (/usr/local/lib/libepetra.a(Epetra_dcrsmv.f.o)) was built for newer macOS version (12.4) than being linked (12.0)
ld: warning: object file (/usr/local/lib/libepetra.a(Epetra_dcrssm.f.o)) was built for newer macOS version (12.4) than being linked (12.0)
ld: warning: object file (/usr/local/lib/libepetra.a(Epetra_dcrssv.f.o)) was built for newer macOS version (12.4) than being linked (12.0)
ld: warning: object file (/usr/local/lib/libaztecoo.a(az_f_util.f.o)) was built for newer macOS version (12.4) than being linked (12.0)
ld: warning: object file (/usr/local/lib/libaztecoo.a(az_reorder.f.o)) was built for newer macOS version (12.4) than being linked (12.0)
Undefined symbols for architecture x86_64:
  "Thyra::ModelEvaluatorBase::DerivativeSupport::description() const", referenced from:
      Thyra::ModelEvaluatorBase::OutArgs<double>::assert_supports(Thyra::ModelEvaluatorBase::EOutArgsDfDp, int, Thyra::ModelEvaluatorBase::Derivative<double> const&) const in libnox.a(NOX_Thyra_Group.C.o)
      Thyra::ModelEvaluatorBase::OutArgs<double>::assert_supports(Thyra::ModelEvaluatorBase::EOutArgsDgDx_dot, int, Thyra::ModelEvaluatorBase::Derivative<double> const&) const in libnox.a(NOX_Thyra_Group.C.o)
      Thyra::ModelEvaluatorBase::OutArgs<double>::assert_supports(Thyra::ModelEvaluatorBase::EOutArgsDgDx, int, Thyra::ModelEvaluatorBase::Derivative<double> const&) const in libnox.a(NOX_Thyra_Group.C.o)
      Thyra::ModelEvaluatorBase::OutArgs<double>::assert_supports(Thyra::ModelEvaluatorBase::EOutArgsDgDp, int, int, Thyra::ModelEvaluatorBase::Derivative<double> const&) const in libnox.a(NOX_Thyra_Group.C.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

My cmake arguments are exactly as given in the guide to build Trilinos-12-12-1 and Xyce for Mac Homebrew.

Thanks,

-Harsha

[xyce-users]

This is puzzling.  Though you say you have invoked CMake with the exact arguments, your build of NOX (from Trilinos) includes files related to Thyra that following our instructions would not have included.  So something is amiss here.

Could you please post the exact cmake invocation you used to build trilinos?  There must be something about it that differs from our recommendation.

[Harsha H S]

This is my cmake invocation for Trilinos

build % cat reconfigure
#!/bin/bash
SRCDIR=$HOME/Trilinos-trilinos-release-12-12-1
ARCHDIR=$HOME/XyceLibs/Parallel
FLAGS="-O3 -fPIC"
cmake \
-G "Unix Makefiles" \
-DCMAKE_C_COMPILER=mpicc \
-DCMAKE_CXX_COMPILER=mpic++ \
-DCMAKE_Fortran_COMPILER=mpif77 \
-DCMAKE_CXX_FLAGS="$FLAGS" \
-DCMAKE_C_FLAGS="$FLAGS" \
-DCMAKE_Fortran_FLAGS="$FLAGS" \
-DCMAKE_INSTALL_PREFIX=$ARCHDIR \
-DCMAKE_MAKE_PROGRAM="make" \
-DTrilinos_ENABLE_NOX=ON \
  -DNOX_ENABLE_LOCA=ON \
-DTrilinos_ENABLE_EpetraExt=ON \
  -DEpetraExt_BUILD_BTF=ON \
  -DEpetraExt_BUILD_EXPERIMENTAL=ON \
  -DEpetraExt_BUILD_GRAPH_REORDERINGS=ON \
-DTrilinos_ENABLE_TrilinosCouplings=ON \
-DTrilinos_ENABLE_Ifpack=ON \
-DTrilinos_ENABLE_Isorropia=ON \
-DTrilinos_ENABLE_AztecOO=ON \
-DTrilinos_ENABLE_Belos=ON \
-DTrilinos_ENABLE_Teuchos=ON \
-DTrilinos_ENABLE_COMPLEX_DOUBLE=ON \
-DTrilinos_ENABLE_Amesos=ON \
 -DAmesos_ENABLE_KLU=ON \
-DTrilinos_ENABLE_Amesos2=ON \
 -DAmesos2_ENABLE_KLU2=ON \
 -DAmesos2_ENABLE_Basker=ON \
-DTrilinos_ENABLE_Sacado=ON \
-DTrilinos_ENABLE_Stokhos=ON \
-DTrilinos_ENABLE_Kokkos=ON \
-DTrilinos_ENABLE_Zoltan=ON \
-DTrilinos_ENABLE_ALL_OPTIONAL_PACKAGES=OFF \
-DTrilinos_ENABLE_CXX11=ON \
-DTPL_ENABLE_AMD=ON \
-DAMD_LIBRARY_DIRS="/usr/local/lib" \
-DTPL_AMD_INCLUDE_DIRS="/usr/local/include/" \
-DTPL_ENABLE_BLAS=ON \
-DTPL_ENABLE_LAPACK=ON \
-DTPL_ENABLE_MPI=ON \
$SRCDIR

--
You received this message because you are subscribed to a topic in the Google Groups "xyce-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/xyce-users/PoZ0yUrNGm8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to xyce-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xyce-users/268a0d8b-2082-4ff3-9b70-a86a4eb51401n%40googlegroups.com.

[xyce-users]

That is indeed puzzling, since there shouldn't be anything in that invocation that causes NOX to be built with Thyra support.  In none of the cases we have ever had in serial or parallel has this support been built into our libraries when we build Trilinos 12.12 on Mac with clang or any other compiler.  But somehow *your* Trilinos build has NOX_Thyra_Group.C getting compiled in.  That doesn't make sense.

I note that your linker errors all show things like this:

     ld: warning: object file (/usr/local/lib/libepetra.a(Epetra_dcrsmm.f.o)) was built for newer macOS version (12.4) than being linked (12.0)

This tells me that there is an install of Trilinos in /usr/local/lib --- and your CMake invocation clearly tells CMake to install *your* build of Trilinos into $HOME/XyceLibs/Parallel.

That suggests to me that somehow you have a *different* version of trilinos installed as well, possibly through a Homebrew package, and our configure script is finding it instead of the one in your ARCHDIR.

This is a recipe for problems.  Are you using Trilinos for some other purpose than Xyce?  If not, then removing the Homebrew-installed Trilinos package would be the simplest fix.

[Harsha H S]

Thanks,

 Removing the version of Trilinos in /usr/local/lib helped solve this issue. Also Xyce  looked into /usr/local/lib for boost and other libraries which conflicted with Trilinos libraries.

Thanks a lot,

-Harsha

To view this discussion on the web visit https://groups.google.com/d/msgid/xyce-users/ee4a1ec1-dc97-4747-aa4e-6a44163e9ba9n%40googlegroups.com.

[xyce-users]

If removing the homebrew-installed Trilinos is not acceptable, there may be another fix.

Our configure is, for whatever reason, currently set up to search your specified "ARCHDIR" after searching all other library directories.  This is probably a mistake on our part --- we are setting up the search for include files so that ARCHDIR is searched FIRST, and we should be doing the same for libraries.  In this case, we are probably finding the headers for your custom-built Trilinos and the libraries that Homebrew installed.

The fix would be a one line change to the file "configure.ac" in the Xyce source tree, followed by re-running bootstrap and reconfiguring/rebuilding Xyce from the beginning.

If you would like to try that instead of removing the homebrew-installed Trilinos, edit configure.ac and find the lines:

   if test -d $ARCHDIR/lib; then
    LDFLAGS="$LDFLAGS -L$ARCHDIR/lib"
   fi

These should probably be:

   if test -d $ARCHDIR/lib; then
    LDFLAGS="-L$ARCHDIR/lib $LDFLAGS"
   fi

If you make that change and it works for you, please let us know and we'll change it on our end.

It is surprising that this has never come up before, since this error has been in configure for many years.

[Harsha H S]

This works for me.

Thanks,

-Harsha

To view this discussion on the web visit https://groups.google.com/d/msgid/xyce-users/7c6ec568-8ac5-45dd-8445-0b8dcff5ff08n%40googlegroups.com.

[xyce-users]

Thank you for letting us know.  We've already made the fix on our end, and it'll be in github by morning.