pkg-config and prefixes

[Allan Wirth]

Hi there!

Recently, I successfully ported some old code of mine using emscripten. The code relied on libgd with libjpeg and libpng (which requires zlib) support, so I had to compile and link all of that in. I didn't understand how to use emscripten ports with libraries, so I decided to do it all from scratch.

This ended up being kind of a mess, because emconfigure seems to override some important variables, like PKG_CONFIG_LIBDIR. I had expected to just build each lib and install the artifacts into a prefix for linking later, but this didn't work directly. You can find my compile script (in the form of a Dockerfile) at https://github.com/allanlw/fractal-web/blob/master/Dockerfile and you will see that it is quite messy, and calls bash inside of emconfigure to re-set important environment variables.

I couldn't find much documentation about emconfigure. Is there a cleaner way to do this?

Thanks,
Allan

[Alon Zakai]

Yeah, this isn't documented very well, sorry about that. get_building_env in tools/shared.py is where we set env vars like PKG_CONFIG_LIBDIR etc. We override them because we can't use the system installed versions, we need emcc-compiled bitcode. So you can in principle alter the pkg-config pointed at there for your ported projects (looks like it points to system/lib/pkgconfig). But I don't know enough about pkg-config to tell how practical that is to do.

The bigger question is what is the best way to do cross-compiling. Open to suggestions here. In general we recommend using ports, which usually amounts to just adding -s USE_ZLIB=1 to CFLAGS, for example. Otherwise, people need to set up CFLAGS to find headers in the right places, etc., somewhat manually, and also manually link in all the bitcode at the end. When you compile multiple new libraries yourself as you describe, perhaps there's a way we could make it easier? What do you mean by "install into a prefix"?

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Brion Vibber]

On Mon, Dec 19, 2016 at 4:44 PM, Alon Zakai <alon...@gmail.com> wrote:

Yeah, this isn't documented very well, sorry about that. get_building_env in tools/shared.py is where we set env vars like PKG_CONFIG_LIBDIR etc. We override them because we can't use the system installed versions, we need emcc-compiled bitcode. So you can in principle alter the pkg-config pointed at there for your ported projects (looks like it points to system/lib/pkgconfig). But I don't know enough about pkg-config to tell how practical that is to do.

The bigger question is what is the best way to do cross-compiling. Open to suggestions here. In general we recommend using ports, which usually amounts to just adding -s USE_ZLIB=1 to CFLAGS, for example. Otherwise, people need to set up CFLAGS to find headers in the right places, etc., somewhat manually, and also manually link in all the bitcode at the end.

pkg-config basically gives you the CFLAGS, LDFLAGS, etc to use with a library, so it should indeed be possible to make .pc entries that return the '-s USE_ZLIB=1' etc. I'll do a little research, could make my life easier to move some of my dependencies to ports. :)

 

When you compile multiple new libraries yourself as you describe, perhaps there's a way we could make it easier? What do you mean by "install into a prefix"?

autotools-style packages expect to be installed somewhere for actual use, either in a public place like /usr or /usr/local or in a private location, passed in like so:

  ./configure --prefix="$BUILDROOT/libs" && make && make install

Under the usual Unix-y 'includes', 'lib', etc subdirectories you get the actual headers, bitcode .a/.so/.dylib output, etc.

In ogv.js, I build all my library dependencies and install them into a common prefix in the build tree this way. When compiling a further dependency or the final build product, I sometimes have to pass those dirs in via -L, -I, PKG_CONFIG_LIBDIR, etc, so their various configuration scripts can all find each other -- for instance libvorbis requires libogg, so their existing configure scripts need to be able to speak a common directory.

This is basically the same thing I'd do if building for Linux if I have a local set of build dependencies (not wanting to be dependent on what the base distribution provides), so it's not *super*-hard compared to other types of cross-compilation. But the less you have to manually deal with building extra dependencies the better...

I find myself wondering if the ports thingy could be generalized a bit to allow applications to add their own library ports without polluting the global namespace, and to be friendlier to pkg-config and other auto-detection mechanisms... I'll take a look.

-- brion

On Sun, Dec 18, 2016 at 8:02 AM, Allan Wirth <all...@gmail.com> wrote:

Hi there!

Recently, I successfully ported some old code of mine using emscripten. The code relied on libgd with libjpeg and libpng (which requires zlib) support, so I had to compile and link all of that in. I didn't understand how to use emscripten ports with libraries, so I decided to do it all from scratch.

This ended up being kind of a mess, because emconfigure seems to override some important variables, like PKG_CONFIG_LIBDIR. I had expected to just build each lib and install the artifacts into a prefix for linking later, but this didn't work directly. You can find my compile script (in the form of a Dockerfile) at https://github.com/allanlw/fractal-web/blob/master/Dockerfile and you will see that it is quite messy, and calls bash inside of emconfigure to re-set important environment variables.

I couldn't find much documentation about emconfigure. Is there a cleaner way to do this?

Thanks,
Allan

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Allan Wirth]

Hmm,

So I had two separate-ish dependency management issues:

1) How can I use emscripten ports with emconfigure. This isn't
described in the documentation, and was why I compiled libpng and zlib
manually. For my application I still would have needed to do libjpeg
and libgd, but this would have simplified it some. I think this one is
probably possible, but I just missed how.

2) What is the expected workflow for using dependencies that aren't
provided by emscripten ports, especially if some of them transitively
do depend on ones that are provided by ports.

For the second issue, the "natural" way to cross compile nontrivial
programs like this would be to set up a prefix, importantly containing
an include, lib and lib/pkgconfig, and then compile each dependency
and install it into the prefix. The configure scripts (or cmake or
whatever) could then be all pointed to that prefix, and would
automatically find how to link against each other, etc. At the end
you'd have a bitcode file (instead of an ELF), and then would just
have to add one final step to compile this to javascript.

Obviously with emscripten there are still going to be some hiccups,
but it would provide a more intuitive experience for folks that are
used to cross-compilation or other compilation involving non-system
libs.

Anyway, just wanted to bring it to your attention/open up the issues
for discussion. I was able to get my stuff successfully working, it
was just kind of bumpy :)

Cheers,
Allan

>>> email to emscripten-disc...@googlegroups.com.

>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to emscripten-disc...@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>
> --

> You received this message because you are subscribed to a topic in the
> Google Groups "emscripten-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/emscripten-discuss/cqYZCu5ELh0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> emscripten-disc...@googlegroups.com.