Minimal deal.ii application loads many surprising dynamic libraries on macos

[Jack Coughlin]

Hello,

I don't know enough to say whether this is an observation about deal.ii or about Mac OSX (or maybe homebrew?), but the compiled binary of step-1 loads around 100 system dynamic libraries:

```

$ DYLD_PRINT_LIBRARIES=1 ./step-1.release

....

```

This includes a bunch of things that seemingly shouldn't be required, such as `libAudioStatistics.dylib`, etc.

The reason I care about this is that I think it adds around .25 seconds to the application startup time, which is a very minor quality of life hit.

In fact, I can't see how to create a dealii binary that starts in less than around a quarter second:

```

minimal.cc:

int main() {
    return 0;
}

CMakeLists:

add_executable(minimal minimal.cc)

deal_ii_setup_target(minimal)

$ time builds/jackc-macos/minimal

builds/jackc-macos/minimal  0.24s user 0.01s system 98% cpu 0.255 total

```

As I say, I don't understand the issue well enough to know if there is anything I or the deal.ii developers can do about it! In any case, 0.25s is a very small price to pay for so much functionality and expressiveness.

Thank you,

-Jack

[Wolfgang Bangerth]

On 4/18/24 23:29, Jack Coughlin wrote:
>
> As I say, I don't understand the issue well enough to know if there is
> anything I or the deal.ii developers can do about it! In any case, 0.25s is a
> very small price to pay for so much functionality and expressiveness.

Jack,
I appreciate the qualification in the last sentence :-)

I don't have a Mac, so can't repeat the experiment and tell you what these
~100 libraries are, but I've looked into this on my Linux laptop. There,
libdeal_II.g.so directly references the following 54 libraries:

Part of Trilinos:
libnox.so.13
libamesos2.so.13
libtacho.so.13
libml.so.13
libifpack.so.13
libamesos.so.13
libaztecoo.so.13
libtrilinosss.so.13
libtpetra.so.13
libtpetraclassicnodeapi.so.13
libepetraext.so.13
libzoltan.so.13
libepetra.so.13
libkokkoskernels.so.13
libteuchoscomm.so.13
libteuchosparameterlist.so.13
libteuchoscore.so.13
libkokkoscore.so.13
libsuperlu_dist.so.5

HDF5:
libhdf5.so.103

Part of OpenCASCADE:
libTKBool.so.11
libTKBRep.so.11
libTKernel.so.11
libTKG3d.so.11
libTKGeomAlgo.so.11
libTKGeomBase.so.11
libTKMath.so.11
libTKMesh.so.11
libTKShHealing.so.11
libTKTopAlgo.so.11
libTKXSBase.so.11
libTKIGES.so.11
libTKSTEP.so.11
libTKSTL.so.11

PETSc and SLEPc:
libslepc.so.3.16
libpetsc.so.3.16
libmetis.so

MPI:
libmpi_cxx.so.40
libmpi.so.40

SUNDIALS:
libsundials_idas.so.4
libsundials_arkode.so.4
libsundials_kinsol.so.5
libsymengine.so.0.8

BLAS and LAPACK:
liblapack.so.3
libblas.so.3

p4est:
libp4est.so.0
libsc.so.0

System and compiler libraries:
libgmp.so.10
libz.so.1
libm.so.6
libstdc++.so.6
libgcc_s.so.1
libc.so.6
ld-linux-x86-64.so.2

These are all libraries from which deal.II directly utilizes functionality. It
is so many because some of the packages we rely on (notably Trilinos and
OpenCASCADE) split their functionality into many separate libraries for
reasons unknown to me.

Beyond these 54, every executable then pulls in another 27 libraries. These
are libraries that, directly or indirectly, one of the 57 above references.
For me, these are:

More from Trilinos:
libgaleri-epetra.so.13
libHYPRE-2.23.0.so
libkokkoscontainers.so.13
libteuchosnumerics.so.13
libteuchosparser.so.13
libteuchosremainder.so.13
libthyracore.so.13
libtriutils.so.13

More from OpenCASCADE:
libTKBO.so.11
libTKG2d.so.11
libTKPrim.so.11
libTKSTEP209.so.11
libTKSTEPAttr.so.11
libTKSTEPBase.so.11

More from PETSc:
libparmetis.so

More from MPI:
libmpi_mpifh.so.40
libopen-pal.so.40
libopen-rte.so.40

More compiler libraries:
libgfortran.so.5
libgomp.so.1
libhwloc.so.15
libquadmath.so.0

Unclear to me:
libevent_core-2.1.so.7
libevent_pthreads-2.1.so.7
libmvec.so.1
libudev.so.1
linux-vdso.so.1 (0x00007ffc27ddc000)


The example you show, libAudioStatistics.dylib, would fall into this last
category. It would be interesting to see which of the dependencies pulls it
in. In general, though, from the example above, nearly everything I see is
there for a fairly good reason.

I will note that you can very substantially slim down if you trim all of the
dependencies. You can build deal.II without reference to Trilinos, PETSc, and
a lot of other stuff, and it compiles substantially faster and is
substantially smaller. It will of course also have substantially less
functionality :-)

Best
W.

--
------------------------------------------------------------------------
Wolfgang Bangerth email: bang...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

[Jack Coughlin]

Wolfgang,

Thank you for the reply, I'm glad to see that the developers are keeping an eye on dependencies and startup time.

In my case, I was actually careful to compile deal.ii ONLY against MPI, so there are only seven direct deps, mostly from my homebrew installation of openmpi. But after figuring out the correct tooling to do this on macos and digging a little more, it seems that openmpi's libhwloc is the culprit. Once it pulls in Foundation.framework and IOKit.framework, we are really off to the races.

It appears that my ubuntu build in a docker container starts up in about half the time (also just built against MPI).

-Jack

[Wolfgang Bangerth]

On 4/19/24 16:11, Jack Coughlin wrote:
>
> In my case, I was actually careful to compile deal.ii ONLY against MPI, so
> there are only seven direct deps, mostly from my homebrew installation of
> openmpi. But after figuring out the correct tooling to do this on macos and
> digging a little more, it seems that openmpi's libhwloc is the culprit. Once
> it pulls in Foundation.framework and IOKit.framework, we are really off to the
> races.

Ah, very interesting! Of course, I have no idea how to address this :-)

[blais...@gmail.com]

Dear Jack,

Interesting you took a look at this. I was always wondering why lethe takes forever to start on my ARM M1 Mac compared to on a regular Linux desktop. Seems you figured it out...

However, I am unsure of the solution for this.

Orthogonally to this, have you ever managed to have cmake ctest command work correctly with dylib files under this setup?

Best

Bruno

[Matthias Maier]

Can one of you test the following for me?

1/ reduce the main.cc to a single "int main() { return 0; }".

2/ Force as needed:

diff --git a/examples/step-1/CMakeLists.txt b/examples/step-1/CMakeLists.txt
index 1a2602ee0d..aa3c6c60be 100644
--- a/examples/step-1/CMakeLists.txt
+++ b/examples/step-1/CMakeLists.txt
@@ -36,4 +36,5 @@ endif()

deal_ii_initialize_cached_variables()
project(${TARGET})
+string(APPEND DEAL_II_LINKER_FLAGS " -Wl,--as-needed")
deal_ii_invoke_autopilot()


Does this link at all? If so, how many libraries are linked in?


MM