Compile once, run everywhere (on Linux)

[paulc...@gmail.com]

Hi all,

 

I've been upgrading my IDE to use wxwidgets 2.9.5 and, since wxwidgets of this version is not available as a Linux package, would need to compile and package it myself. As I've been doing this, I thought it would be great to have one package I could run on as many Linux distros as possible (32bit/64bit question aside), so I decide to take a stub at this. Based on my testing it seems like this may work, but I applied several "hacks" related to libpng that I'd like to review here.

 

First step was to compile with sufficiently old libc. I compiled on Ubuntu 10.04, which allows me to use the binaries on recent Debian, Ubuntu, Fedora 18, and ArchLinux Linuxes.

 

Second step was to decide what to do with libpng. The problem with libpng on Linux is that GTK (also) depends on libpng and pulls in its own version. So, there are three options:

 

1. Not include libpng at all; it will still be pulled in by GTK, but none of the PNG-related code is compiled in, so wxwidgets doesn't know anything about PNG type at all. Not a good option for me.

2. Use builtin library. This discussion (https://groups.google.com/forum/?fromgroups=#!topic/wx-users/2QWQD0qUVUc) as well as this ticket (http://trac.wxwidgets.org/ticket/14157) provide some details on why this is not a good idea. This is where I started and while it seemed to work on systems with libpng 1.2, the app was seg faulting on systems with libpng 1.5 (and the builtin version is also 1.5).

3. Reference system library. This works, but only as long as the version of libpng this is linked against is available. So, if I build on Ubuntu 10.04, which has libpng 1.2.42, it won't run on Fedora 18 or ArchLinux, which both have libpng 1.5.x as wx.so is looking for libpng 1.2. It's possible to add a symlink to libpng.so, trying to trick wx.so into using 1.5 libpng, but unfortunately all the symbols are linked against PNG12_0 version, so this doesn't work either.

 

At this point I felt stuck, but decided to try Vadim's suggestion from the earlier discussion about building with libpng=builtin, but not linking it (and using --allow-shlib-undefined, but this wasn't needed as this seems to be a default setting). This worked, as all png symbols were shown as unresolved and without any version numbers, which was exactly what I wanted. (I also used zlib=sys as it would be pulled in by system libpng.)

 

With that change I could use the generated binaries, because GTK would pull in libpng and my application would simply use it. Unfortunately, this wasn't the end of it and two more changes were needed.

 

When wxwidgets is compiled with libpng=builtin, it stores the version of libpng it's packaged with, which in my case 1.5.x. When it runs on a system with any other minor version (png only checks for maj/min version numbers), it complaints that the versions are different and refuses to do any useful work. This is where my first "fix" comes in. I replaced '#define PNG_LIBPNG_VER_STRING "1.5.7"' with '#define PNG_LIBPNG_VER_STRING (png_get_header_ver(NULL))' to always pass the check. I know, this is not ideal, but if I *do* want to run on systems with different libpng versions, this is exactly what I need. And I also tested libpng 1.2 to 1.5 to be compatible enough for my purposes.

 

The binaries now worked as expected on 32bit systems. But there was still a problem on 64bit Linuxes. The images would not load and I was getting "Couldn't load a PNG image - file is corrupted or not enough memory." messages. When I checked what was causing this, it turned out that png_get_IHDR( png_ptr, info_ptr, &width, &height, &bit_depth, &color_type, &interlace_type, NULL, NULL ); call in wxPNGHandler::LoadFile was returning zero width (the height was correct). This was puzzling as exactly the same code was working correctly on 32bit system. I suspected some sort of data truncation, but couldn't figure out what could be causing it.

 

Then I looked at libpng headers. LoadFile from wxwidgets defines width as png_uint_32, but its definition has changed between libpng 1.2 I was using it with and libpng 1.5 it was compiled with. In libpng 1.2 it's defined as

 

typedef unsigned long png_uint_32;
typedef long png_int_32;

but in libpng 1.5 it's defined as

 

#if defined(INT_MAX) && (INT_MAX > 0x7ffffffeL)
typedef unsigned int png_uint_32;
typedef int png_int_32;
#else
typedef unsigned long png_uint_32;
typedef long png_int_32;
#endif

This means that libpng 1.2 attempts to return width as unsigned long on 64bit system, but wxwidgets code only expects unsigned int. My "fix" was to disable the check and always use unsigned long, which allowed LoadFile (and SaveFile) code to work with both 1.2 and 1.5 versions of libpng.

 

Now, I'm not absolutely sure about my two libpng "fixes" and would welcome feedback on why this may be wrong or what a better way to do this may be. This has been tested so far on 32bit Ubuntu 10.04 and 12.04, Fedora 18 and ArchLinux, as well as on 64bit Ubuntu 10.04, 12.04 and Fedora 18 without any noticeable issues (with libpng 1.2 and libpng 1.5).

 

Paul.

 

P.S. for those interested, the IDE is available here (http://studio.zerobrane.com) and 2.9.5 upgrade changes are discussed in this ticket (https://github.com/pkulchenko/ZeroBraneStudio/issues/89).

[Fulvio Senore]

I build on an old ubuntu version too, and I configure to use the builtin
libpng.

As you noticed the program won't work on distros that use a newer
version of libpng, but for me the problem has always been solved by
installing the libpng1.2 package. It should be available to all
distributions and it looks like this solution fixes the problem.

Fulvio Senore

> --
> Please read http://www.wxwidgets.org/support/mlhowto.htm before posting.
>
> To unsubscribe, send email to wx-users+u...@googlegroups.com
> or visit http://groups.google.com/group/wx-users
>
>

[paulc...@gmail.com]

Hi Fulvio,

 

> As you noticed the program won't work on distros that use a newer
> version of libpng, but for me the problem has always been solved by
> installing the libpng1.2 package. It should be available to all

You said you used builtin version, but it's 1.5. Why would you need to install libpng 1.2? Or did you mean you used system libpng (and installed libpng 1.2 on newer systems as it would then be required)?

 

This may be a good way if it works; I've been looking for something that can work out of the box and wasn't sure if I run into conflicts with both libpng 1.2 and libpng 1.5 loaded. Did you get a chance to test your solution on 32bit and 64bit systems?

 

Paul.

[Fulvio Senore]

I mean building on a system with libpng1.2.
If the user has a distro with a different version of libpng he should
install the libpng1.2 package.

I did not test this solution by myself, but users reported that the
program worked. Sorry I don't know if they were using 32 or 64 bit.

Fulvio

Il 20/03/2013 19.14, paulc...@gmail.com ha scritto:
> Hi Fulvio,
> > As you noticed the program won't work on distros that use a newer
> > version of libpng, but for me the problem has always been solved by
> > installing the libpng1.2 package. It should be available to all
> You said you used builtin version, but it's 1.5. Why would you need to
> install libpng 1.2? Or did you mean you used system libpng (and
> installed libpng 1.2 on newer systems as it would then be required)?
> This may be a good way if it works; I've been looking for something that
> can work out of the box and wasn't sure if I run into conflicts with
> both libpng 1.2 and libpng 1.5 loaded. Did you get a chance to test your
> solution on 32bit and 64bit systems?
> Paul.
>
> On Wednesday, March 20, 2013 10:54:19 AM UTC-7, fsenore wrote:
>
> I build on an old ubuntu version too, and I configure to use the
> builtin
> libpng.
>
> As you noticed the program won't work on distros that use a newer
> version of libpng, but for me the problem has always been solved by
> installing the libpng1.2 package. It should be available to all
> distributions and it looks like this solution fixes the problem.
>
> Fulvio Senore
>

> <https://groups.google.com/forum/?fromgroups=#!topic/wx-users/2QWQD0qUVUc>)

>
> > as well as this ticket (http://trac.wxwidgets.org/ticket/14157

> <http://trac.wxwidgets.org/ticket/14157>) provide

> <https://github.com/pkulchenko/ZeroBraneStudio/issues/89>).

> >
> > --
> > Please read http://www.wxwidgets.org/support/mlhowto.htm
> <http://www.wxwidgets.org/support/mlhowto.htm> before posting.
> >
> > To unsubscribe, send email to wx-users+u...@googlegroups.com

> <javascript:>
> > or visit http://groups.google.com/group/wx-users
> <http://groups.google.com/group/wx-users>

[Fulvio Senore]

I have just noticed that I wrote that I configure to use the builtin
libpng. Sorry, I made a mistake. I should never try doing things in a
hurry...

What I meant is that I configure to use the system libpng in a computer
with version 1.2. The resulting program works on computers with 1.2.

In distributions that use a different version of libpng users need to
install the libpng1.2 package and the program is reported to work.

Sorry for the mistake.

Fulvio

[paulc...@gmail.com]

Hi Fulvio,

 

> What I meant is that I configure to use the system libpng in a computer
> with version 1.2. The resulting program works on computers with 1.2.

Right, that's what I was getting at with my response; thank you for clarifying.

 

The reason I asked about 64bit is that it seems like you'd still run into the same issue I had with not working on 64bit systems as your library would be using libpng 1.2, but would be compiled with libpng 1.5 headers (even when you link against a system library, wxwidgets still uses headers from libpng 1.5 that ship with it as far as I understand; I see ./src/png included before /usr/include/libpng12).

 

Paul.

[Vadim Zeitlin]

On Wed, 20 Mar 2013 16:16:05 -0700 (PDT) John Bowler wrote:

JB> On Wednesday, March 20, 2013 10:14:16 AM UTC-7, paulc...@gmail.com wrote:
JB> >
JB> > this ticket (http://trac.wxwidgets.org/ticket/14157)
JB> >
JB>
JB> I updated the ticket to explain how to fix the issue. A general solution
JB> requires 1.6 (or, to put it another way, fixing it with 1.5 requires a lot
JB> of work to do the same thing as is fully supported in 1.6)

Interesting, I didn't know about this new release. Somewhat confusingly,
there is no mention of it at http://www.libpng.org/pub/png/libpng.html
neither, it's only available from http://libpng.sourceforge.net/

JB> It's almost certainly a whole lot easier just to have the ticket fixed so
JB> that you can use the built-in version.

It does look relatively easy to do what you propose and I'll try to do it
before 3.0 but any help with it would still be very much appreciated, so if
you or anybody else affected by this issue could please try updating our
libpng version and patching configure as recommended, it would be very
welcome.

TIA,
VZ

--
TT-Solutions: wxWidgets consultancy and technical support
http://www.tt-solutions.com/

[stevec...@gmail.com]

Hi Guys,

> I've been upgrading my IDE to use wxWidgets 2.9.5 and,
> since wxWidgets of this version is not available as a

> Linux package, would need to compile and package it
> myself. As I've been doing this, I thought it would be
> great to have one package I could run on as many Linux
> distros as possible (32bit/64bit question aside), so I
> decide to take a stub at this. Based on my testing it
> seems like this may work, but I applied several "hacks"
> related to libpng that I'd like to review here.

I believe this is what Citrus Perl does.

As the name suggests, it's a perl distribution but it includes
Wx::Perl and wxWidgets.

It says it includes compiled versions of 2.8.x and 2.9.x
here:

http://sourceforge.net/projects/citrusperl/files/Perl%205.16.1/Linux/

It may help resolve some of your issues.

Good luck.

Steve.

[paulc...@gmail.com]

Hi Vadim,

 

> you or anybody else affected by this issue could please try updating our
> libpng version and patching configure as recommended, it would be very

Other than configure changes, what code changes would be involved? As far as I understand, at the very minimum, all png calls in imagpng.cpp file would need to be prefixed with the same prefix as configured for builtin libpng.

 

Are there any ways that a call from GTK may end up in builtin png library? Or png data structures created by the builtin library get used by the system library? For example, when something like ArtProvider is used?

 

Paul.

[John Bowler]

On Thursday, March 21, 2013 9:39:35 AM UTC-7, paulc...@gmail.com wrote:

Other than configure changes, what code changes would be involved? As far as I understand, at the very minimum, all png calls in imagpng.cpp file would need to be prefixed with the same prefix as configured for builtin libpng.

 

It just works; if the *libpng* configure script is invoked with --with-libpng-prefix=wx_ then png.h will #define every API symbol appropriately.

 

John Bowler

[Vadim Zeitlin]

On Thu, 21 Mar 2013 09:39:35 -0700 (PDT) wrote:

> > you or anybody else affected by this issue could please try updating our
> > libpng version and patching configure as recommended, it would be very
> Other than configure changes, what code changes would be involved?

The included library should be updated to 1.6 (using the standard svn
vendor branch update procedure).

> As far as I understand, at the very minimum, all png calls in imagpng.cpp
> file would need to be prefixed with the same prefix as configured for
> builtin libpng.

According to John this is not the case as the renaming #defines are
emitted by configure.

> Are there any ways that a call from GTK may end up in builtin png library?

Not as far as I know.

> Or png data structures created by the builtin library get used by the
> system library?

Again, no, I don't think so.

Regards,

[paulc...@gmail.com]

Hi John,

 

> It just works; if the *libpng* configure script is invoked with --with-libpng-prefix=wx_ then png.h will #define every API symbol appropriately.

 

Great! So for me to try this, I'd get libpng1.6, run configure on it with --with-libpng-prefix=wx_, replace the src png/ folder with the new one, and build wxwidgets as I normally do (using libpng=builtin), right?

 

Paul.

[John Bowler]

On Thu, Mar 21, 2013 at 11:05 AM, <paulc...@gmail.com> wrote:
> Great! So for me to try this, I'd get libpng1.6, run configure on it with
> --with-libpng-prefix=wx_, replace the src png/ folder with the new one, and
> build wxwidgets as I normally do (using libpng=builtin), right?

run configure, then *build* and *install* to your private (built-in)
png/ directory (I guess --prefix=$PWD/png or something like that to
configure), and, of course, make sure you are getting *that* set of
header files. I recommend hacking '#error WRONG HEADER' into
/usr/include/png.h to detect errors (possibly within #ifdef
WX_SOMETHING).

I looked at gtk (2.0 and 3.0) - well, I looked at *every* header file
on my system (it's Gentoo, so I have all the headers). Gtk does not
have any references to png_ (case insensitive) so the API is clean -
no part of the libpng API is exposed in the Gtk API. The following
headers show problems:

boost: it looks like it has inlined methods that expose libpng
functionality and they are visible in /usr/include.
libmng: not surprising, it's heavily dependent on libpng and basically
just considerably extends the API

wx itself shows some matches on my search, but they're all safe -
local #defines or enum values that happen to have PNG_ embedded

John Bowler

[Paul K]

Hi John,

> run configure, then *build* and *install* to your private (built-in)
> png/ directory (I guess --prefix=$PWD/png or something like that to
> configure), and, of course, make sure you are getting *that* set of

I gave this a quick try and it appears to work. This is what I did.

1. Get libpng 1.6 and unzip it
2. ./configure --with-libpng-prefix=wxpng_
3. make
4. don't run "make install" but simply copy the folder with libpng
instead of src/png folder.

Then configure and run wxwidgets build as before.

If I do "make install" (and I included proper --prefix), it installs 4
folders (bin/, lib/, include/, and share/), which is not what we need.
If I don't do make, then some files (like pnglibconf.h) are missing.

I built and ran it on a system with libpng 1.2 and also ran it on a
system with libpng 1.5. The built image includes the prefixes as
configured.

Paul.

[John Bowler]

On Thu, Mar 21, 2013 at 1:06 PM, Paul K <paulc...@gmail.com> wrote:
> If I do "make install" (and I included proper --prefix), it installs 4
> folders (bin/, lib/, include/, and share/), which is not what we need.
> If I don't do make, then some files (like pnglibconf.h) are missing.

Looking at src/png in wxGTK 2.9.4.1 I see that it does have
pnglibconf.h, but it does not have Makefile.

Since pnglibconf.h is made by Makefile something has to be wrong.

It seems to me that the build dependencies are wrong; when
-with-libpng=builtin is passed to the top level configure then
src/libpng should both be configured and make-ed before anything in
the rest of wxGTK that depends on libpng. The configure would
generate Makefile and the subsequent make would generate pnglibconf.h
and the static library.

I believe if the configure/make steps happen in the correct
(dependency) order then everything will work for a static (definitely
not dynamic) built-in libpng.

John Bowler <jbo...@acm.org>