the neverending png odyssey

[mabshoff]

While poking around some matplotlib related blowup around libpng and
frameworks I noticed the following:

We have two sets of dynamic libpngs:

On Linux:

-rw-r--r-- 1 mabshoff mabshoff 689468 2009-02-03 19:30 local/lib/
libpng12.a
-rwxr-xr-x 1 mabshoff mabshoff 839 2009-02-03 19:30 local/lib/
libpng12.la
lrwxrwxrwx 1 mabshoff mabshoff 18 2009-02-03 19:30 local/lib/
libpng12.so -> libpng12.so.0.22.0
lrwxrwxrwx 1 mabshoff mabshoff 18 2009-02-03 19:30 local/lib/
libpng12.so.0 -> libpng12.so.0.22.0
-rwxr-xr-x 1 mabshoff mabshoff 496727 2009-02-03 19:30 local/lib/
libpng12.so.0.22.0
lrwxrwxrwx 1 mabshoff mabshoff 10 2009-02-03 19:30 local/lib/
libpng.a -> libpng12.a
lrwxrwxrwx 1 mabshoff mabshoff 11 2009-02-03 19:30 local/lib/
libpng.la -> libpng12.la
lrwxrwxrwx 1 mabshoff mabshoff 11 2009-02-03 19:30 local/lib/
libpng.so -> libpng12.so
lrwxrwxrwx 1 mabshoff mabshoff 16 2009-02-03 19:30 local/lib/
libpng.so.3 -> libpng.so.3.22.0
-rwxr-xr-x 1 mabshoff mabshoff 512975 2009-02-03 19:30 local/lib/
libpng.so.3.22.0

and on OSX:

-rwxr-xr-x 1 michaelabshoff staff 246992 Feb 3 22:26 libpng.
3.22.0.dylib
lrwxr-xr-x 1 michaelabshoff staff 19 Feb 3 22:26 libpng.
3.dylib -> libpng.3.22.0.dylib
lrwxr-xr-x 1 michaelabshoff staff 10 Feb 3 22:26 libpng.a ->
libpng12.a
lrwxr-xr-x 1 michaelabshoff staff 14 Feb 3 22:26 libpng.dylib -
> libpng12.dylib
lrwxr-xr-x 1 michaelabshoff staff 11 Feb 3 22:26 libpng.la ->
libpng12.la
-rwxr-xr-x 1 michaelabshoff staff 249196 Feb 3 22:26
libpng12.0.22.0.dylib
lrwxr-xr-x 1 michaelabshoff staff 21 Feb 3 22:26
libpng12.0.dylib -> libpng12.0.22.0.dylib
-rw-r--r-- 1 michaelabshoff staff 703408 Feb 3 22:26 libpng12.a
lrwxr-xr-x 1 michaelabshoff staff 21 Feb 3 22:26
libpng12.dylib -> libpng12.0.22.0.dylib
-rwxr-xr-x 1 michaelabshoff staff 862 Feb 3 22:26 libpng12.la

The code that blows up is linked against libpng, not libpng12, so if
we delete libpng.* and meddle a little in various configure and
makefiles (think "sed s/libpng\./libpng12\./g" we might fix this
dreaded problem once and for all. I might have time to poke around
this next week, but until then: Thoughts?

Cheers,

Michael

[mabshoff]

mabshoff wrote:
> While poking around some matplotlib related blowup around libpng and
> frameworks I noticed the following:

<SNIP>

> makefiles (think "sed s/libpng\./libpng12\./g" we might fix this
> dreaded problem once and for all. I might have time to poke around
> this next week, but until then: Thoughts?

And after removing libpng.* on OSX and the following patch

diff -r 177c872eb17a module_list.py
--- a/module_list.py Thu Feb 05 05:04:30 2009 -0800
+++ b/module_list.py Thu Feb 05 17:39:00 2009 -0800
@@ -581,7 +581,7 @@

Extension('sage.matrix.matrix_mod2_dense',
sources = ['sage/matrix/matrix_mod2_dense.pyx'],
- libraries = ['gmp','m4ri', 'png', 'gd']),
+ libraries = ['gmp','m4ri', 'png12', 'gd']),

Extension('sage.matrix.matrix_modn_dense',
sources = ['sage/matrix/matrix_modn_dense.pyx'],

and a rebuild of that extension doctests still do pass for that file.
The above extension was the initial reason to start building dynamic
libpngs on OSX, so if we can sort out the other build issues we should
be good to go.

Notice that this also offers the possibility to reenable png support
for R and Python since we currently disabled it.

> Cheers,
>
> Michael

Cheers,

Michael

[mabshoff]

Ok, I think I finally fixed the problem:

* delete libpng.*, that leaves libpng12.*
* doctests pass on OSX, code like dvipng that is linked against the
IOKit framework works now that didn't work when libpng.* existed.
* fix build issues, for example the new matplotlib.spkg, the Sage
library and a bunch of others. The first two are one line patches, for
some of the others I expect more work, but we might do such things on
OSX only

With the above we are finally getting rid of a long standing problems
and it turns out to be much less pain that I thought it would be :). I
need to verify that all of this works on OSX 10.4, but I am confident
it will with minimal changes.

The above work will likely not be in 3.3, but should make it into 3.4
[.x].

Cheers,

Michael

[mhampton]

Three cheers! Hooray! Hooray! Hooray!

I think I've had things related to this crop up several times, and I
am glad you have resolved it (crossing my fingers!).

-Marshall

[mabshoff]

On Feb 6, 1:47 am, mhampton <hampto...@gmail.com> wrote:
> Three cheers! Hooray! Hooray! Hooray!
>
> I think I've had things related to this crop up several times, and I
> am glad you have resolved it (crossing my fingers!).
>
> -Marshall

Well, we are not quite there yet, but if you don't want to do any more
development on a Sage install at least on OSX 10.5 you just delete
libpng.* and all is well, i.e. Sage passes its doctests and you can
execute any code that is linked against frameworks without penalty.

Building in that mode is a different story and upgrading will fail
unless you patch various packages that expect libpng to be there, but
as I mentioned I started sorting this out, so hopefully 3.4 will see
this issue dealt with once and for all. I have other various large
projects going for 3.4 that didn't make it into 3.3, so we shall see
how this works out. This project is less of a priority than the ecl
switch, the SSE2 only build for disting and the quadratic forms code
we need in Sage for SD 13, so I need to get sufficiently bored or
frustrated by these other projects to spend time on the libpng
issue :)

Cheers,

Michael

[mabshoff]

On Feb 6, 2:49 am, mabshoff <mabsh...@googlemail.com> wrote:
> On Feb 6, 1:47 am, mhampton <hampto...@gmail.com> wrote:
>
> > Three cheers! Hooray! Hooray! Hooray!
>
> > I think I've had things related to this crop up several times, and I
> > am glad you have resolved it (crossing my fingers!).
>
> > -Marshall
>
> Well, we are not quite there yet, but if you don't want to do any more
> development on a Sage install at least on OSX 10.5 you just delete
> libpng.* and all is well, i.e. Sage passes its doctests and you can
> execute any code that is linked against frameworks without penalty.

Ok, since this issue is impeding the needed bits for the new multiedge
plotting code for graphs due to dependencies on the MPL.spkg update I
took a gamble and it turned out that my guess was correct: We need
three trivial patches and an spkg update. Since I am updating the
linpng.spkg anyway in 3.3.rc1 I might as well do the other three
fixes.

For the record, but I will open the three remaining tickets in the
next couple hours.

Cheers,

Michael

libpng12 Changes

* delete libpng.* on $SAGE_LOCAL/lib/ in the libpng.spkg

* patch matplotlib's libpng detection code to

diff -r e9c8b372cab7 patches/setupext.py
--- a/patches/setupext.py Mon Jan 19 06:47:20 2009 -0500
+++ b/patches/setupext.py Fri Feb 13 03:48:59 2009 -0800
@@ -574,7 +574,7 @@
module.include_dirs.append(numpy.get_include())

def add_png_flags(module):
- try_pkgconfig(module, 'libpng', 'png')
+ try_pkgconfig(module, 'libpng12', 'png12')
add_base_flags(module)
add_numpy_flags(module)
module.libraries.append('z')


* path the Sage library's matrix_mod2_dense to link against png12:

# HG changeset patch
# User michael...@Sprocketer.local
# Date 1234523512 28800
# Node ID fc08a7e8da116f9596376a1205670b410ea84145
# Parent 038b84f34083060e79bdfa442493d1b56314b2a3
Link matrix_mod2_dense against libpng12 instead of libpng

diff -r 038b84f34083 -r fc08a7e8da11 module_list.py
--- a/module_list.py Wed Feb 11 00:01:24 2009 -0800
+++ b/module_list.py Fri Feb 13 03:11:52 2009 -0800

@@ -581,7 +581,7 @@

Extension('sage.matrix.matrix_mod2_dense',
sources = ['sage/matrix/matrix_mod2_dense.pyx'],
- libraries = ['gmp','m4ri', 'png', 'gd']),
+ libraries = ['gmp','m4ri', 'png12', 'gd']),

Extension('sage.matrix.matrix_modn_dense',
sources = ['sage/matrix/matrix_modn_dense.pyx'],

* tachyon - in src/unix/Make-config change -lpng to -lpng12

USEPNG= -DUSEPNG
PNGINC= -I$(SAGE_LOCAL)/include
PNGLIB= -L$(SAGE_LOCAL)/lib -lpng12 -lz

We can probably set PNGLIB in spkg-install, so we don't have to edit
any build system files.