LDFLAGS, lib search dir & find_library()

[Noam Meltzer]

Hi,

When I run meson, I want to specify a custom LDFLAGS with arbitrary lib directories.

For example:
LDFLAGS='-L/storage/lib1/build99 -L/storage/lib2/build89' meson ${PWD} build/

However, these flags are not honored by 'find_library()'.
What is the proper procedure for specifying lib directories when executing meson?

Best regards,
Noam Meltzer

[Jussi Pakkanen]

The 100% proper solution is to have your dependencies provide pkg-config files. It might take some effort but once it works it's effortless and the only really reliable way to solve this problem. I highly recommend that you do that if at all possible.

If you can't do that then you can explicitly specify the location where find_library looks for its stuff so something like this:

thelib = find_library('foolib', dirs : ['/path/to/somewhere', '/other/path'])

If you wish to have these settable from the command line, one way is to set an option in meson_options.txt like this:

option('lib1dir', type : 'string', value : '/something/something')

and use them like this:

thelib = find_library('foolib', dirs : get_option('lib1dir'))

Then you can override these by initialising your build directory like this:

meson <srcdir> <builddir> -Dlib1dir=/somewhere/num42

Note that Meson caches successfull find_library detected libraries. If you change the path value afterwards, it will keep using the old one. In this case just nuke your build dir and try it again.

[Noam Meltzer]

On Mon, Jul 27, 2015 at 3:21 PM Jussi Pakkanen <jpak...@gmail.com> wrote:

On Mon, Jul 27, 2015 at 1:55 PM, Noam Meltzer <tsn...@gmail.com> wrote:

 

When I run meson, I want to specify a custom LDFLAGS with arbitrary lib directories.

For example:
LDFLAGS='-L/storage/lib1/build99 -L/storage/lib2/build89' meson ${PWD} build/

However, these flags are not honored by 'find_library()'.
What is the proper procedure for specifying lib directories when executing meson?

The 100% proper solution is to have your dependencies provide pkg-config files. It might take some effort but once it works it's effortless and the only really reliable way to solve this problem. I highly recommend that you do that if at all possible.

pkg-config is the holy grail, however I consider replacing a legacy build system and it might be a problem (for all the wrong reasons).

 

If you can't do that then you can explicitly specify the location where find_library looks for its stuff so something like this:

thelib = find_library('foolib', dirs : ['/path/to/somewhere', '/other/path'])

If you wish to have these settable from the command line, one way is to set an option in meson_options.txt like this:

option('lib1dir', type : 'string', value : '/something/something') 

and use them like this:

thelib = find_library('foolib', dirs : get_option('lib1dir'))

Then you can override these by initialising your build directory like this:

meson <srcdir> <builddir> -Dlib1dir=/somewhere/num42

this is a bit cumbersome & requires specific knowledge of the option names per meson project and not generic (like LDFLAGS, or LD_LIBRARY_PATH for example)

In comparison, CPPFLAGS does easily control the places where to look for header files (CPPFLAGS='-I/some/path')

However, I have a bit different suggestion, hope you will like it:

looking at Environment.find_library() code it seems like it is already doing some assumptions on default library dirs (if 'dirs is None')

I thought of adding a new member to CoreData: lib_searchdir. This member will be set from command line using --lib-searchdir / -L

I can code this patch myself (and send a pull request), if you find it appropriate.

[Jussi Pakkanen]

On Mon, Jul 27, 2015 at 3:55 PM, Noam Meltzer <tsn...@gmail.com> wrote:

 

this is a bit cumbersome & requires specific knowledge of the option names per meson project and not generic (like LDFLAGS, or LD_LIBRARY_PATH for example)

In comparison, CPPFLAGS does easily control the places where to look for header files (CPPFLAGS='-I/some/path')

However, I have a bit different suggestion, hope you will like it:

looking at Environment.find_library() code it seems like it is already doing some assumptions on default library dirs (if 'dirs is None')

I thought of adding a new member to CoreData: lib_searchdir. This member will be set from command line using --lib-searchdir / -L

This answer is going to have two parts. The easy and working but hacky. The second part is the thorough and correct answer that is not very useful immediately.

Starting with the first bit, if you know that you will always have your libraries available you can do the hacky thing. Meson will take LDFLAGS and put them in your link commands. Which means you can do this:

executable('foo', 'foo.c', link_args : '-lyourlibnamehere')

and call Meson like this:

CPPFLAGS=-I/somewhere/inc LDFLAGS=-L/somewhere/lib meson ...

and it will work. The only downside (apart from feeling a bit icky) is that if you got your paths wrong, it will fail during link rather than during configure. Though if you check for headers then it will fail at that point.

The longer version is that the way find_library works is not very satisfactory. Instead of being a standalone function it should be moved to be part of the compiler object just like has_header and all those other ones. The big problem is that there does not seem to be a way to get the compiler to tell where it found the library from, only that it found it (indirectly from the return code).

Trying to get the library paths manually does not work nicely with LDFLAGS either:

LDFLAGS=-L/foo/bar gcc -print-search-dirs | grep foo

returns empty.

I guess one way of dong this would be to make find_library check LDFLAGS and extract any -L flags and use those dirs for searching libraries. That would work most of the time at least.

[Noam Meltzer]

I accidentally sent this in a private reply. Sending to the mailing list + Jussi's reply.

On Mon, Jul 27, 2015 at 5:11 PM Noam Meltzer <tsn...@gmail.com> wrote:

On Mon, Jul 27, 2015 at 4:37 PM Jussi Pakkanen <jpak...@gmail.com> wrote:

On Mon, Jul 27, 2015 at 3:55 PM, Noam Meltzer <tsn...@gmail.com> wrote:

 

this is a bit cumbersome & requires specific knowledge of the option names per meson project and not generic (like LDFLAGS, or LD_LIBRARY_PATH for example)

In comparison, CPPFLAGS does easily control the places where to look for header files (CPPFLAGS='-I/some/path')

However, I have a bit different suggestion, hope you will like it:

looking at Environment.find_library() code it seems like it is already doing some assumptions on default library dirs (if 'dirs is None')

I thought of adding a new member to CoreData: lib_searchdir. This member will be set from command line using --lib-searchdir / -L

This answer is going to have two parts. The easy and working but hacky. The second part is the thorough and correct answer that is not very useful immediately.

Starting with the first bit, if you know that you will always have your libraries available you can do the hacky thing. Meson will take LDFLAGS and put them in your link commands. Which means you can do this:

executable('foo', 'foo.c', link_args : '-lyourlibnamehere')

and call Meson like this:

CPPFLAGS=-I/somewhere/inc LDFLAGS=-L/somewhere/lib meson ...

and it will work. The only downside (apart from feeling a bit icky) is that if you got your paths wrong, it will fail during link rather than during configure. Though if you check for headers then it will fail at that point.

Yes, this is hacky, and I'd rather avoid this.

 

The longer version is that the way find_library works is not very satisfactory. Instead of being a standalone function it should be moved to be part of the compiler object just like has_header and all those other ones. The big problem is that there does not seem to be a way to get the compiler to tell where it found the library from, only that it found it (indirectly from the return code).

Trying to get the library paths manually does not work nicely with LDFLAGS either:

LDFLAGS=-L/foo/bar gcc -print-search-dirs | grep foo

returns empty.

I guess one way of dong this would be to make find_library check LDFLAGS and extract any -L flags and use those dirs for searching libraries. That would work most of the time at least.

I'll try to explain my thoughts on this, but lets try to look at the issue again:

1. We need to know the location of a library.

2. Tools we have to determine the location: pkg-config & looking at a pre-defined list of directories (what find_library() does)

3. pkg-config is not always an option.

So now, we need to find a way to hint meson where to look for a library.

If I would be able to use pkg-config, I can help it by setting PKG_CONFIG_PATH environment variable.

A very similar approach in semantics would be to either set LDFLAGS or LD_LIBRARY_PATH. However, parsing LDFLAGS and/or LD_LIBRARY_PATH is very hacky.

So, what we need is a way to hint meson to look in the right place.

I think that adding a new CLI argument for meson would be an elegant solution for it. Something like:
meson -L/somewhere/lib ...

This approach can be extended and provide a more complete solution for headers (include dirs) as well.

Jussi's reply:

I agree with you that the current solution is not very good and needs improving. However I don't want to add new flags and command line options unless they are really needed. Autotools and CMake both have masses of options that are both completely unnecessary and confusing. It is almost always worth spending some time thinking about if the requirements you have could be solved behind the scenes with existing code and practices than blindly adding new flags.

Could you please file a bug on this? It's better to keep the discussion concentrated there. 

[Noam Meltzer]

Filed issue #217 https://github.com/mesonbuild/meson/issues/217