Duplicated libraries

[Rob McDonald]

I'm using the tip of master 1.4.0.

I'm building with CMake, with FIND_PACKAGE(FLTK CONFIG REQUIRED).

No matter whether I specify my libraries via:

TARGET_LINK_LIBRARIES(foo
${FLTK_LIBRARIES}
)

or

TARGET_LINK_LIBRARIES(foo
fltk::z
fltk::png
fltk::jpeg
fltk::fltk
fltk::forms
fltk::images
fltk::gl

)

I end up with:

[3/5] Linking CXX executable foo

ld: warning: ignoring duplicate libraries: 'fltk/install/lib/libfltk.a', 'fltk/install/lib/libfltk_jpeg.a', 'fltk/install/lib/libfltk_png.a', 'fltk/install/lib/libfltk_z.a'

This seems harmless enough, but it would be great to make it go away.  I've spent the past few hours trying different things, but I can't figure out how CMake is duplicating these libraries.

Is anyone else seeing this sort of thing?

Rob

[Albrecht Schlosser]

On 3/16/24 07:57 Rob McDonald wrote:
> I'm building with CMake, with FIND_PACKAGE(FLTK CONFIG REQUIRED).
>
> No matter whether I specify my libraries via:
>
> TARGET_LINK_LIBRARIES(foo
> ${FLTK_LIBRARIES}
> )

You should not do this since you want to take advantage of modern CMake.
FLTK_LIBRARIES contains *all* libraries generated by FLTK in our current
version of the config module.

The earlier/obsolete FindFLTK (1.1) module allowed to select which
libraries you need by setting some variables. In modern CMake you would
select "components" but this is not implemented in our FLTK build.

> TARGET_LINK_LIBRARIES(foo
> fltk::z
> fltk::png
> fltk::jpeg
> fltk::fltk
> fltk::forms
> fltk::images
> fltk::gl
> )
>

You should not do this (in this form) as well. This would also (try to)
link all available FLTK libs. See below for suggestions.

> I end up with:
>
> [3/5] Linking CXX executable foo
>
> ld: warning: ignoring duplicate libraries:
> 'fltk/install/lib/libfltk.a', 'fltk/install/lib/libfltk_jpeg.a',
> 'fltk/install/lib/libfltk_png.a', 'fltk/install/lib/libfltk_z.a'
>
>

> This seems harmless enough, ... Is anyone else seeing this sort of thing?

Do you see this on macOS, or on which OS?

I've never seen this on macOS but never on Linux, however only with
-lcairo and test/cairo_test.cxx where -lcairo was indeed linked twice. I
could change this by changing test/CMakeLists.txt so libcairo is not
added more than once.

What happens? Modern CMake adds linkage requirements to each library and
it should - IMHO - avoid duplicating libs. However, if you add libs more
than once, then I assume that CMake does it in the linker commandline as
well. That said,

- you should *never* add fltk::z, fltk::png, or fltk::jpeg explicitly
because they will be linked indirectly by fltk::images.

- if you need one or more of fltk::forms, fltk::images, or fltk::gl then
add them to your target_link_libraries() but not fltk::fltk because it
will be included implicitly. Using all three of these libs should be
harmless.

- if you don't use one of the libs above, add *only* fltk::fltk to
target_link_libraries().

The most likely set of libs you will need to use is (I'm guessing)
fltk::images + fltk::gl (if you use OpenGL) but very likely not fltk::forms.

This *should* avoid multiply linking libs and the warning message should
- hopefully - disappear. If this is not the case, then it's either a bug
in FLTK's CMake files or in CMake itself and I would like to see more
details.

You can find out more if you look at the linker command by doing
something like this to avoid too much noise:

1. build/link your app normally
2. delete the executable
3. link your app again in verbose mode, e.g. `cmake --build . --verbose`

The 3rd command should show you only the linker commandline. There are
several other ways to make the build "verbose", e.g. `ninja -v` if you
build with ninja, or `make VERBOSE=1 <target>` if you're using make.

However this might or might not give you a clue *why* a particular lib
is linked more than once.

[Albrecht Schlosser]

On 3/16/24 22:29 'Albrecht Schlosser' wrote:
> I've never seen this on macOS but never on Linux, however only with

Correction, this should read:

I've seen this on macOS but never on Linux, however only with...

> -lcairo and test/cairo_test.cxx where -lcairo was indeed linked twice. I
> could change this by changing test/CMakeLists.txt so libcairo is not
> added more than once.

PS: I'll commit and push this soon but this is not related to your
issue, Rob.

[Rob McDonald]

On Saturday, March 16, 2024 at 2:29:28 PM UTC-7 Albrecht-S wrote:

On 3/16/24 07:57 Rob McDonald wrote:
> I'm building with CMake, with FIND_PACKAGE(FLTK CONFIG REQUIRED).
>
> No matter whether I specify my libraries via:
>
> TARGET_LINK_LIBRARIES(foo
> ${FLTK_LIBRARIES}
> )

You should not do this since you want to take advantage of modern CMake.
FLTK_LIBRARIES contains *all* libraries generated by FLTK in our current
version of the config module.

The earlier/obsolete FindFLTK (1.1) module allowed to select which
libraries you need by setting some variables. In modern CMake you would
select "components" but this is not implemented in our FLTK build.

OK, in old (module based) FindFLTK.cmake, the above seemed to work.

Thanks much for the explanation -- it seems obvious (and I nearly went down that path).  I feel stupid for not knowing this.

> TARGET_LINK_LIBRARIES(foo
> fltk::z
> fltk::png
> fltk::jpeg
> fltk::fltk
> fltk::forms
> fltk::images
> fltk::gl
> )
>

You should not do this (in this form) as well. This would also (try to)
link all available FLTK libs. See below for suggestions.

> This seems harmless enough, ... Is anyone else seeing this sort of thing?

Do you see this on macOS, or on which OS?

I'm diagnosing this on MacOS, but it may happen on other platforms too.

 

I've never seen this on macOS but never on Linux, however only with
-lcairo and test/cairo_test.cxx where -lcairo was indeed linked twice. I
could change this by changing test/CMakeLists.txt so libcairo is not
added more than once.

What happens? Modern CMake adds linkage requirements to each library and
it should - IMHO - avoid duplicating libs. However, if you add libs more
than once, then I assume that CMake does it in the linker commandline as
well. That said,

- you should *never* add fltk::z, fltk::png, or fltk::jpeg explicitly
because they will be linked indirectly by fltk::images.

Where is this documented?  I had assumed that fltk::images contained some common image-related code and that the other three just contained format specific code.  I did not expect that the linker targets would be cumulative like this (to create the possibility of this mistake).

Also -- how can I be sure that it is safe to leave these out?  I probably can (I don't explicitly use any of the FLTK image code), but how can I be sure?  Is setting the operating system icon enough to require this?  Custom widget decorations?  I know I don't use FLTK to open and display images, but there might be some unintended use.

 

- if you need one or more of fltk::forms, fltk::images, or fltk::gl then
add them to your target_link_libraries() but not fltk::fltk because it
will be included implicitly. Using all three of these libs should be
harmless.

Again, where is this documented?  Is fltk::forms only needed for old  Forms compatibility?

OK, I've gone searching more and have found this stuff documented in section 3.1 of README.CMake.txt

It is super interesting that fltk::forms, fltk::images, and fltk::gl will all implicitly add fltk::fltk -- such that adding all three is harmless and adding any of them makes fltk::fltk unnecessary.  However, there are the other prohibitions against multiple adds...

From reading all this, I think I should be able to get away with just 

fltk::gl

And nothing else...

- if you don't use one of the libs above, add *only* fltk::fltk to
target_link_libraries().

The most likely set of libs you will need to use is (I'm guessing)
fltk::images + fltk::gl (if you use OpenGL) but very likely not fltk::forms.

This *should* avoid multiply linking libs and the warning message should
- hopefully - disappear. If this is not the case, then it's either a bug
in FLTK's CMake files or in CMake itself and I would like to see more
details.

BTW, the 'old' FindFLTL.cmake Module approach would implicitly 'Find(OpenGL)' (unless it was explicitly skipped) and would add the resulting OpenGL libraries to the FLTK libraries.

This does not appear to be what fltk::gl does at all -- which is fine, but I guess this is just another implied difference between the old and Modern way.

 

You can find out more if you look at the linker command by doing
something like this to avoid too much noise:

1. build/link your app normally
2. delete the executable
3. link your app again in verbose mode, e.g. `cmake --build . --verbose`

The 3rd command should show you only the linker commandline. There are
several other ways to make the build "verbose", e.g. `ninja -v` if you
build with ninja, or `make VERBOSE=1 <target>` if you're using make.

However this might or might not give you a clue *why* a particular lib
is linked more than once.

This is generally how I have been diagnosing this sort of thing already.  However, this does not tell me that fltk::images includes all of fltk::jpeg and fltk::png  -- and other implied rules like that.

Thanks for the help,

Rob

 

[Albrecht Schlosser]

On 3/16/24 23:43 Rob McDonald wrote:

On Saturday, March 16, 2024 at 2:29:28 PM UTC-7 Albrecht-S wrote:

The earlier/obsolete FindFLTK (1.1) module allowed to select which
libraries you need by setting some variables. In modern CMake you would
select "components" but this is not implemented in our FLTK build.

OK, in old (module based) FindFLTK.cmake, the above seemed to work.

That's because none of these libs *implied* linking any other lib. You had to know which one you need(ed) to link, or FLTK had to list all dependencies in this variable. The new, modern CMake approach is that every single lib "knows" its own dependencies (aka linkage requirements) and provides you (i.e. your build procedure) with all required include directories and other compiler flags etc. etc. - that's what "modern CMake" does.

Thanks much for the explanation -- it seems obvious (and I nearly went down that path).  I feel stupid for not knowing this.

No problem ...

Do you see this on macOS, or on which OS?

I'm diagnosing this on MacOS, but it may happen on other platforms too.

I believe that only the macOS linker issues a warning about this. Generally this should be harmless anyway.

Modern CMake adds linkage requirements to each library and
it should - IMHO - avoid duplicating libs. However, if you add libs more
than once, then I assume that CMake does it in the linker commandline as
well. That said,

- you should *never* add fltk::z, fltk::png, or fltk::jpeg explicitly
because they will be linked indirectly by fltk::images.

Where is this documented?

Good question. Thanks for asking. If this is not clearly documented I need to improve the docs. [see below]

I had assumed that fltk::images contained some common image-related code and that the other three just contained format specific code.  I did not expect that the linker targets would be cumulative like this (to create the possibility of this mistake).

I understand "cumulative" but I'm not sure what you mean. Let me rephrase this: The term used in CMake is that each library *target* contains its own linkage requirements. Hence fltk::images has the linkage requirements that it needs fltk::fltk, fltk:jpeg and fltk::png. fltk::png requires fltk::z, fltk::fltk requires X11 and/or Wayland libs, libcairo, libfontconfig, ... All this adds libs recursively to the linkage requirements that result in the linker commandline generated by CMake.

Also -- how can I be sure that it is safe to leave these out?

You can be sure that every library target brings its own linkage requirements with it. So the only question is: which library do you really use, i.e. whose library's functions do you call directly?

I probably can (I don't explicitly use any of the FLTK image code), but how can I be sure?

Simple answer: trial and error. Try the minimal list and build your project. If it works you're done, otherwise the linker will spit out undefined symbols that give you a clue which other library you need.

More detailed answers: for instance, if you call fl_register_images() you request functions of the image library fltk::images. If you use any of the Fl::xxx_Image classes (except Fl_RGB_Image and maybe a few others) you need fltk::images, etc.

Is setting the operating system icon enough to require this?  Custom widget decorations?  I know I don't use FLTK to open and display images, but there might be some unintended use.

As said above, if you don't know what you need, just try without fltk::images and you'll see.

- if you need one or more of fltk::forms, fltk::images, or fltk::gl then
add them to your target_link_libraries() but not fltk::fltk because it
will be included implicitly. Using all three of these libs should be
harmless.

Again, where is this documented?  Is fltk::forms only needed for old  Forms compatibility?

Basically yes, but if you happen to use e.g. Fl_Timer (an old Forms compatibility widget) you need fltk::forms which is not that obvious but also not very likely (the docs discourage its usage). There may be more unexpected things that are in the forms library but usually you won't use such code in "normal" FLTK applications. Again, the linker will tell you.

OK, I've gone searching more and have found this stuff documented in section 3.1 of README.CMake.txt

OK, I see. I thought I documented it but wasn't 100% sure. This looks pretty complete, are there any more questions?

It is super interesting that fltk::forms, fltk::images, and fltk::gl will all implicitly add fltk::fltk -- such that adding all three is harmless and adding any of them makes fltk::fltk unnecessary.  However, there are the other prohibitions against multiple adds...

Note that multiple adds should never be harmful. A warning (as from the macOS linker) can indicate something but you can certainly live with it (I would try to eliminate the warning though, same as what I see you doing).

From reading all this, I think I should be able to get away with just 

fltk::gl

And nothing else...

Yep, if you are using OpenGL (Fl_Gl_Window). If you need more the linker will tell you.

BTW, the 'old' FindFLTK.cmake Module approach would implicitly 'Find(OpenGL)' (unless it was explicitly skipped) and would add the resulting OpenGL libraries to the FLTK libraries.

This does not appear to be what fltk::gl does at all -- which is fine, but I guess this is just another implied difference between the old and Modern way.

Hmm, I'm not sure I understand you. If you link your program with fltk::gl this *should* imply all required OpenGL libs to satisfy the linker. If this is not happening then this is a bug in FLTK's CMake support. I'll check this ... tomorrow.

Or, if I misunderstood you, please clarify what you mean. Do you need to link OpenGL system libs in your project's CMake files explicitly?

...  this does not tell me that fltk::images includes all of fltk::jpeg and fltk::png  -- and other implied rules like that.

Simply put, there's only one rule: every CMake library target contains all of its linkage requirements (recursively) - and again, if this is not the case then it's a bug, please let me know.

Thanks for the help,

Thanks for testing and feedback, I appreciate your constructive help very much.

Albrecht