install problems with clang@12.0.0 and Xcode12.0 from spack

[Alexander Greiner]

Hi,

I'm new here (and also relatively new to deal.II). I'm trying to install dea...@9.2.0 from spack, running into errors. These errors are not directly linked to deal.II and therefore I'm not sure if this is the right place to post my problem. Unfortunately, cl...@12.0.0 and Xcode12.0 seem to be quite new and there is not a lot to find in the www... Furthermore, I'm using g...@10.2.0.

spack install dealii

1) Thereby, installing OpenBlas leads to the error:

Error: ProcessError: Command exited with status 2: 'make' '-j12'

Then, I tried to install the most recent develop version of OpenBlas -- with success. And continue with:

spack install dealii ^openblas@develop

2) Thereby, installing arpack-ng leads again to:

Error: ProcessError: Command exited with status 2: 'make' '-j12'

Then, I tried to install the most recent develop version of arpack-ng -- with success. And continue with:

spack install dealii ^openblas@develop ^arpack-ng@develop

3) Thereby, installing netlib-scalapack leads again to:

Error: ProcessError: Command exited with status 2: 'make' '-j12'

Unfortunately, there is no newer version of this package.

Now my question, has anyone else tried to install dealii 9.2.0 and macOS Catalina with cl...@12.0.0 and Xcode12.0? Or has anyone suggestions how to overcome these issues? I'm looking forward to any help and ideas! Let me know if you need more detailed information.

Kind regards 

Alex

[Alexander Greiner]

Update: 

I managed to came as far as spack at least tries to install deal.II using the following command:

spack install dealii~scalapack ^openblas@develop ^arpack-ng@develop ^trilinos~mumps

I have no idea whether this makes sense or not, but it was the only way to get that far. Installing deal.II I received the following error message again: 

Error: ProcessError: Command exited with status 2: 'make' '-j12'

I would like to show you the spack build log, but I can't find that file. Actually, spack does not really abort...just 'strange' things happen that I do not understand. I'll attach the terminal output. In the end I have to abort the process manually since (even after hours) nothing more than what you can see in the terminal output happens.

Again I appreciate any help and ideas!

Alex

[Bruno Turcksin]

Alex,

Try using less processors. Something like:

spack install -j2 dealii~scalapack ^openblas@develop ^arpack-ng@develop ^trilinos~mumps

Best,

Bruno

[Alexander Greiner]

Hi Bruno,

thank you for your response. Unfortunately, I am still receiving the 'same' error message. This time:

Error: ProcessError: Command exited with status 2: 'make' '-j2'
Are there any other suggestions what I could try?

Regards,

Alex

[Wolfgang Bangerth]

On 9/26/20 6:21 AM, 'Alexander Greiner' via deal.II User Group wrote:
>
> thank you for your response. Unfortunately, I am still receiving the 'same'
> error message. This time:
> Error: ProcessError: Command exited with status 2: 'make' '-j2'
> Are there any other suggestions what I could try?

The real error will have shown up higher up in the output. Can you scroll up a
few pages to see whether there are other errors you can see? Is there a log
file in the directory in which you run all of this?

Best
W.

--
------------------------------------------------------------------------
Wolfgang Bangerth email: bang...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

[Alexander Greiner]

Hi Wolfgang,

at least I could not find a log file, but your hint encouraged me to find another solution. So I ran spack with -d and -v options to print out more information directly to the terminal. As you predicted, there was an error (attached) that seems to be related to the ginkgo package. I left out ginkgo and was finally able to install deal.II 9.2 with the following command:

spack -d -v install dealii~scalapack~ginkgo ^openblas@develop ^arpack-ng@develop ^trilinos~mumps

Now, trying to run step-1 tutorial, I encounter a problem that some (quite a lot) libraries are not linked properly (previously discussed here: https://groups.google.com/g/dealii/c/hKNu1IfwClw). Error message is attached. I executed:

cd examples/step-1

spack load dealii

cmake .

make run

I don't know whether this problem is related to deal.II or to spack. But maybe there is a fix to that, without adapting all dynamic library files manually? I've also attached the summary.log and detailed.log in case it might be helpful.

Thank's a lot so far!

Regards 

Alex

[Praveen C]

Try adding path to you spack lib, like this

export DYLD_FALLBACK_LIBRARY_PATH=/opt/spack/lib

I add this to my $HOME/.profile file.

Best
praveen

[Praveen C]

I am not sure my solution will work in your case.

I use spack view to install symlinks into /opt/spack

Then I compile deal.II myself using dependencies installed in /opt/spack

Setting DYLD path to /opt/spack/lib then works for me, otherwise I too get those library not found errors when running deal.ii codes.

In the past I never had to set DYLD when using spack to install deal.II dependencies, but in some recent xcode update, the behaviour seems to have changed.

Perhaps in your case this might work.

export DYLD_FALLBACK_LIBRARY_PATH=`spack location -i trilinos`/lib

Does this now find libmuelu… ?

Best

praveen

[Alexander Greiner]

Hi Praveen, 

"Stupid" question beforehand: Do I have to reinstall deal.II? If yes, you can skip the remainder. If no, I did the following:

I added that line to my ./bash_profile opened a new terminal window and executed

source ./bash_profile

spack load dealii

cd spack/opt/.../deal.II/examples/step-1

cmake .   (I removed all previously generated "make"-files to obtain the initial directory content of step-1)

make run

Unfortunately, libmuelu is still not found. I attached a printout of my environment variables, if this might help...

[Praveen C]

No, you dont have to recompile deal.II for what I am suggesting.

I suppose that lib is installed by spack, have you checked

ls `spack location -i trilinos`/lib/libmuelu*

Not sure if this will help, but 

Have you enabled Full Disk Access for Terminal.app

If not, can you enable it and try. See this

https://macperformanceguide.com/blog/2020/20200208_2004macOS-Catalina-add-fullDiskAccess-Terminal.html

best

praveen

[Alexander Greiner]

Hi, thanks for your suggestions.

Yes, Trilinos is installed by spack and 

ls `spack location -i trilinos`/lib/libmuelu*

shows me the correct paths to the libmuelu libraries. 

I also enabled Full Disk Access for Terminal.app and restarted the App as described. 

Still, the libraries are not linked properly.

Best 

Alex

[luca.heltai]

My suggestion is to install also environment-modules, and then use

spack load dealii

This will popolate DYLD_LIBRARY_PATH with the correct values and allows your cmake to find all libraries.

Unfortunately, recent updates to spack have made its functionality a bit clumsy in Mac OS...

L.

> --
> The deal.II project is located at http://www.dealii.org/
> For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to dealii+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/97294b8d-4815-4f96-a346-8d8d6037ff35n%40googlegroups.com.

[Alexander Greiner]

Hi Luca,

I installed environment-modules and currently have the following lines in my .bash_profile:

### dealii 9.2 ###

SPACK_ROOT=/Users/alexandergreiner/spack

export PATH=$SPACK_ROOT/bin:$PATH

MODULES_HOME=$(spack location -i environment-modules)

source ${MODULES_HOME}/init/zsh

. $SPACK_ROOT/share/spack/setup-env.sh

After spack load dealii, spack find --loaded shows (attached).

But, echo $DYLD_LIBRARY_PATH shows nothing. Do I have to pay attention to something due to its clumsiness? Is there any further helpful information/files that I can provide to you?

I highly appreciate the help I am receiving from all of you!

Regards

Alex

[Luca Heltai]

Dear Alex, 

what happens after loading dealii with `spack load dealii`, if you try to build (from scratch) `step-40`? 

Can you send us the output of  cmake and make?

Luca.

To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/661b4cf3-dd9d-46ff-9d15-9cc6f074290en%40googlegroups.com.

[Alexander Greiner]

Hi Luca,

I ran step-40 and the output of Cmake log file from the CMakeFiles Folder is attached. Additionally, the output of make -d if that is what you meant. 

Thanks! 

Alex

[Praveen C]

I notice that you are using cmake from /opt/local/bin, but spack would have installed its own cmake. Could you try using cmake from spack to compile and then run the example ?

best

praveen

[Alexander Greiner]

Hi Praveen,

I've (hopefully) done what you suggested and attached the outputs again. It has definitely changed something, but I'm still running into the same error.

Might it be possible that during installation also not spack's Cmake version was used resulting in missing links in dylib-files?

Regards

Alex

[Alexander Greiner]

Dear all,

I would like to share a bit more information what I found out (as a beginner) that might be helpful. I used otool -L and otool -l to check out the .dylib files. I attached the results for libdeal_II.9.2.0.dylib and libdeal_II.g.9.2.0.dylib. 

Here I could find out that libraries from the suite-sparse package are not linked correctly. Although the rpath is correct, the load command is missing an @rpath (see e.g. libumfpack.5.dylib). This problem exists also in other dependent libraries like libmuelu from trilinos where libraries from suite-sparse are not linked properly.

I furthermore attached the output of otool -L and otool -l  applied to the step-40 executable. Here occurs the same problem with suite-sparse again, and additionally, there are no rpaths set such it cannot find the libmuelu-adapters.13.dylib for example (among all others from the trilinos package).

Hope that helps to clarify my issues! Thank's!

Alex

[Jean-Paul Pelteret]

Hi Alexander,

I am, unfortunately, marginally acquainted with the problem that you’re facing. I’m not 100% sure, but I believe that this is an upstream problem that has to do with how these packages are configured for Apple systems. I too noticed that within a couple of update cycles SuiteSparse, for example, went from linking fine, to not specifying any path (only a library name) when linked, to linking with an @rpath placeholder. Here is a silent update that offered some mitigation for the problem with SuiteSparse, which probably (?) was released after the Mac package was built:

https://github.com/DrTimothyAldenDavis/SuiteSparse/commit/f900df09d2ee3f7ecfd9243b69e181d80f720790#diff-d1c5d2f7b51daaca451f6322f8cd89adR59

This is a super annoying problem. So what can you do about it? Well, possibly the quickest solution to your problem lies with a tool called “install_name_tool”, which allows you to modify the paths of the libraries that an executable is linked against. I’ve attached a script that I wrote for myself a while ago because, well, I faced the same issue. If you modify it appropriately, you can at least use it to build your executable and modify the locations of the linked libraries before you get to run the program. It’s not a great solution, and you definitely shouldn’t have to go this route, but maybe it would at least help you get going while we (or the dependency maintainers) get to fixing the core issue.

Jean-Paul

[Alexander Greiner]

Hi Jean-Paul,

thank you, thank you, thank you! I finally got it working! 

You're script worked with a few adaptions because I'm not using spack views. So I had to copy and paste quite many libraries, set the paths correctly, and then run it in several library directories of different packages where the links were not correct. 

Side note: 

To get the script running I had to change the file format to UNIX. Therefore, open the script with vi and enter :set fileformat=unix in the vi command mode. Then save it :wq!

Again, thank you all a lot for your patience and helpfulness! I appreciate it!

Alex

[Praveen C]

Here is an observation.

I set 

DYLD_FALLBACK_LIBRARY_PATH=/path/to/spack/view/lib 

and run as

./step-40

then it runs fine.

But if I use “make run”, then it cannot find libumfpack !!!

Best

praveen