difference in behaviour of "sage -python blah.py" vs. "sage and then load('blah.py')"

[Dima Pasechnik]

Running sage's python on a python file gives ImportError:

sage subshell$ sage -python lp.py
Traceback (most recent call last):
File "lp.py", line 3, in <module>
from cvxopt import matrix
File "/home/dima/sage/sage-4.5.1/local/lib/python2.6/site-packages/
cvxopt/__init__.py", line 30, in <module>
import base
ImportError: /home/dima/sage/sage-4.5.1/local/lib/liblapack.so:
undefined symbol: __powidf2

--------------------------
on the other hand, load'ing this file in sage happily works:

sage subshell$ sage
----------------------------------------------------------------------
| Sage Version 4.5.1, Release Date: 2010-07-19 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
sage: load('lp.py')
pcost dcost gap pres dres k/t
0: -8.1000e+00 -1.8300e+01 4e+00 0e+00 8e-01 1e+00
[....]

This all happens on skynet's taurus:

sage subshell$ uname -a
Linux taurus 2.6.32.12-115.fc12.x86_64 #1 SMP Fri Apr 30 19:46:25 UTC
2010 x86_64 x86_64 x86_64 GNU/Linux

with Sage compiled using gcc-4.5.0

I was unable to reproduce this on systems that run gcc-4.x with x<5.

Any ideas where to look?

Thanks,
Dima

[Harald Schilly]

On 2 Aug., 13:48, Dima Pasechnik <dimp...@gmail.com> wrote:
> Any ideas where to look?
>

Can you look up what LD_LIBRARY_PATH says inside the sage shell (sage -
sh) and when running -python lp.py ?

import os; os.environ['LD_LIBRARY_PATH']

Maybe that's different.

H

[Dima Pasechnik]

I just found that linking in addition against libgcc_s.so cures the
problem.
This is suspiciously similar to (presumably) freebsd-specific lapack
trouble discussed in
http://wiki.sagemath.org/freebsd/sage-4.5

So this seems to be a proof that this is rather gcc-4.5-related,
rather than FreedBSD-specific?!

DIma

[Dr. David Kirkby]

Have you tried gcc 4.5.1? It could be a new bug in 4.5.0, which if you are lucky
will be corrected in 4.5.1.

Note that wheres 4.5.0 had extra features compared to the 4.4 series, 4.5.1 is
just a bug-fix release on 4.5.0, so it should have fewer bugs.

Dave

[Dima Pasechnik]

Dave,
I don't have gcc-4.5.1 installed anywhere, not yet at least.
You can try this yourself ( first without the 1-line change below,
then with:
install the cvxopt-1.1.2.spkg from the link I posted on #6456,
run spkg-check, first without the 1-line change below, then with)

diff -r 117baef5ef34 patches/setup.py
--- a/patches/setup.py Sun Aug 01 11:48:42 2010 -0700
+++ b/patches/setup.py Mon Aug 02 09:53:17 2010 -0400
@@ -13,7 +13,7 @@
libraries =
['m','lapack','gsl','gslcblas','blas','cblas','atlas']
else:
# libraries = ['m','lapack','blas','cblas','atlas','gfortran']
- libraries =
['m','lapack','gsl','gslcblas','blas','cblas','gfortran','atlas']
+ libraries =
['m','lapack','gsl','gslcblas','blas','cblas','gfortran','atlas','gcc_s']

Dima

[Dima Pasechnik]

Dave,

It looks more like a linker bug/feature --- the same behaviour on
skynet's eno (Linux), but
no errors on skynet's mark (Solaris, which I believe uses Sun's
linker).

Dima


On Aug 2, 3:08 pm, "Dr. David Kirkby" <david.kir...@onetel.net> wrote:

[Willem Jan Palenstijn]

On Mon, Aug 02, 2010 at 08:33:49AM -0700, Dima Pasechnik wrote:
> Dave,
>
> It looks more like a linker bug/feature --- the same behaviour on
> skynet's eno (Linux), but
> no errors on skynet's mark (Solaris, which I believe uses Sun's
> linker).

Is it a fresh build or an upgrade of sage?

-Willem Jan

[Dima Pasechnik]

a fresh build on taurus.
On mark and eno I used builds by John Palmieri, so I don't know about
them.

Dima

[Dr. David Kirkby]

On 08/ 2/10 02:58 PM, Dima Pasechnik wrote:
> Dave,
> I don't have gcc-4.5.1 installed anywhere, not yet at least.
> You can try this yourself ( first without the 1-line change below,
> then with:
> install the cvxopt-1.1.2.spkg from the link I posted on #6456,
> run spkg-check, first without the 1-line change below, then with)

I don't have 4.5.1 installed yet either.

I installed 4.5.0 a few days back, only to find 4.5.1 was released a few hours
after I installing 4.5.0!

Dave

[Dima Pasechnik]

I am currently building the atlas spkg with (appropriately adjusted)
patch on
#9600, so that libgcc_s is linked in, and see if it fixes the trouble.

Actually, it looks like a blocker for the release...

Dima

[Dima Pasechnik]

I wasn't able to figure out how to make the appropriate change to
ATLAS spkg using that patch.
Perhaps someone more knowledgeable about ATLAS can have a go at it.

Dima

[John H Palmieri]

They were fresh builds. On eno, could it be an issue with
LD_LIBRARY_PATH? I know that it gets unset when you run screen (on
eno). Have you checked that?

--
John

[Peter Jeremy]

On 2010-Aug-02 05:41:22 -0700, Dima Pasechnik <dim...@gmail.com> wrote:
>I just found that linking in addition against libgcc_s.so cures the
>problem.
>This is suspiciously similar to (presumably) freebsd-specific lapack
>trouble discussed in
>http://wiki.sagemath.org/freebsd/sage-4.5

And trac #9600 (as has been discovered).

>So this seems to be a proof that this is rather gcc-4.5-related,
>rather than FreedBSD-specific?!

I had recorded this as FreeBSD-specific because it presumably worked
on Linux. As I noted, it is possible it is an issue with the way
atlas is built in Sage.

I suspect it's not directly gcc 4.5 related but rather gcc 4.5 shows
the problem by producing code that needs libgcc helper functions.

--
Peter Jeremy

[Peter Jeremy]

On 2010-Aug-02 06:58:10 -0700, Dima Pasechnik <dim...@gmail.com> wrote:
>install the cvxopt-1.1.2.spkg from the link I posted on #6456,
>run spkg-check, first without the 1-line change below, then with)
>
>diff -r 117baef5ef34 patches/setup.py
>--- a/patches/setup.py Sun Aug 01 11:48:42 2010 -0700
>+++ b/patches/setup.py Mon Aug 02 09:53:17 2010 -0400
>@@ -13,7 +13,7 @@
> libraries =
>['m','lapack','gsl','gslcblas','blas','cblas','atlas']
> else:
> # libraries = ['m','lapack','blas','cblas','atlas','gfortran']
>- libraries = ['m','lapack','gsl','gslcblas','blas','cblas','gfortran','atlas']
>+ libraries = ['m','lapack','gsl','gslcblas','blas','cblas','gfortran','atlas','gcc_s']

I would not recommend this fix as it will be needed wherever liblapack
is used (and will not work if there are any cases where liblapack is
dlopen()d. I believe the better fix is to add a dependency on libgcc_s
to liblapack.so as per my fix in #9600.

--
Peter Jeremy

[Willem Jan Palenstijn]

I'm trying to reproduce this, but on skynet's taurus I don't even get a
liblapack.so, but only liblapack.a. (The build isn't done yet, but the lapack
package is done.)

Am I missing something? I checked, but none of my recent sage builds on
multiple machines seem to have liblapack.so.

-Willem Jan

[Dima Pasechnik]

On Aug 2, 9:23 pm, Peter Jeremy <peterjer...@acm.org> wrote:

yes, that's exactly what I was trying to figure out, but failed how to
do this properly.
(and my internet access and thus gcc-4.5 access it very limited these
weeks)

It would be great if you can post such a patch somewhere.

Dima

> --
> Peter Jeremy
>
>  application_pgp-signature_part
> < 1KViewDownload

[Dima Pasechnik]

my taurus build of sage 4.5.1 has liblapack.so just fine...

On Aug 2, 10:11 pm, Willem Jan Palenstijn <w...@usecode.org> wrote:
> On Tue, Aug 03, 2010 at 05:23:17AM +1000, Peter Jeremy wrote:

[John H Palmieri]

On Aug 2, 1:11 pm, Willem Jan Palenstijn <w...@usecode.org> wrote:
> On Tue, Aug 03, 2010 at 05:23:17AM +1000, Peter Jeremy wrote:

I bet at the end of the ATLAS log they all say

ld: cannot find -lgfortran

when trying to build the shared libraries. This happens for me on
sage.math, for example. It doesn't seem to hurt anything, as far as I
can tell, but I could be wrong about this...

--
John

[Willem Jan Palenstijn]

On Mon, Aug 02, 2010 at 01:28:36PM -0700, Dima Pasechnik wrote:
> my taurus build of sage 4.5.1 has liblapack.so just fine...

Ah, now I notice it's generated by atlas. (Sorry, I stupidly missed that rather
obvious bit of #9600). Atlas seems to use ld directly to link liblapack.a into
liblapack.so. If you use gcc as frontend for the linker, it should
automatically link in the required missing functions like __powidf2. It
shouldn't be necessary to add gcc_s manually.

-Willem Jan

> On Aug 2, 10:11?pm, Willem Jan Palenstijn <w...@usecode.org> wrote:
> > On Tue, Aug 03, 2010 at 05:23:17AM +1000, Peter Jeremy wrote:
> > > On 2010-Aug-02 06:58:10 -0700, Dima Pasechnik <dimp...@gmail.com> wrote:
> > > >install the cvxopt-1.1.2.spkg from the link I posted on #6456,
> > > >run spkg-check, first without the 1-line change below, then with)
> >
> > > >diff -r 117baef5ef34 patches/setup.py

> > > >--- a/patches/setup.py ?Sun Aug 01 11:48:42 2010 -0700
> > > >+++ b/patches/setup.py ?Mon Aug 02 09:53:17 2010 -0400
> > > >@@ -13,7 +13,7 @@
> > > > ? ? libraries =

> > > >['m','lapack','gsl','gslcblas','blas','cblas','atlas']
> > > > else:

> > > > # ? ?libraries = ['m','lapack','blas','cblas','atlas','gfortran']
> > > >- ? ?libraries = ['m','lapack','gsl','gslcblas','blas','cblas','gfortran','atlas']
> > > >+ ? ?libraries = ['m','lapack','gsl','gslcblas','blas','cblas','gfortran','atlas','gcc_s']

> >
> > > I would not recommend this fix as it will be needed wherever liblapack
> > > is used (and will not work if there are any cases where liblapack is

> > > dlopen()d. ?I believe the better fix is to add a dependency on libgcc_s

> > > to liblapack.so as per my fix in #9600.
> >
> > I'm trying to reproduce this, but on skynet's taurus I don't even get a
> > liblapack.so, but only liblapack.a. (The build isn't done yet, but the lapack
> > package is done.)
> >
> > Am I missing something? I checked, but none of my recent sage builds on
> > multiple machines seem to have liblapack.so.
> >
> > -Willem Jan
>

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

[Willem Jan Palenstijn]

On Mon, Aug 02, 2010 at 08:43:53PM +0000, Willem Jan Palenstijn wrote:
> On Mon, Aug 02, 2010 at 01:28:36PM -0700, Dima Pasechnik wrote:
> > my taurus build of sage 4.5.1 has liblapack.so just fine...
>
> Ah, now I notice it's generated by atlas. (Sorry, I stupidly missed that rather
> obvious bit of #9600). Atlas seems to use ld directly to link liblapack.a into
> liblapack.so. If you use gcc as frontend for the linker, it should
> automatically link in the required missing functions like __powidf2. It
> shouldn't be necessary to add gcc_s manually.

Replying to self: I did a quick test, and also added the results to #9600:

[wjp@eno sage-4.5.2.rc0]$ ./sage -python
Python 2.6.4 (r264:75706, Aug 1 2010, 12:24:29)
[GCC 4.5.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cvxopt.base

Traceback (most recent call last):

File "<stdin>", line 1, in <module>
ImportError: /home/wjp/eno/sage-4.5.2.rc0/local/lib/liblapack.so: undefined symbol: __powidf2
>>>
[wjp@eno sage-4.5.2.rc0]$ cd local/lib
[wjp@eno lib]$ gcc -shared -o liblapack.so -Wl,-soname,liblapack.so -Wl,--whole-archive liblapack.a -Wl,--no-whole-archive
[wjp@eno lib]$ cd ../..
[wjp@eno sage-4.5.2.rc0]$ ./sage -python Python 2.6.4 (r264:75706, Aug 1 2010, 12:24:29)
[GCC 4.5.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cvxopt.base
>>>

This is on eno with a sage install copied from
/home/palmiere/eno/sage-4.5.2.rc0 . In my own build of sage-4.5.1 on taurus, I
still don't have a liblapack.so, but the above import works fine there.

I haven't looked into what makes liblapack.so appear/disappear yet. Does anyone
familiar with atlas/lapack know this offhand?

-Willem Jan

[Mitesh Patel]

On 08/02/2010 03:33 PM, John H Palmieri wrote:
>> I'm trying to reproduce this, but on skynet's taurus I don't even get a
>> liblapack.so, but only liblapack.a. (The build isn't done yet, but the lapack
>> package is done.)
>>
>> Am I missing something? I checked, but none of my recent sage builds on
>> multiple machines seem to have liblapack.so.
>
> I bet at the end of the ATLAS log they all say
>
> ld: cannot find -lgfortran
>
> when trying to build the shared libraries. This happens for me on
> sage.math, for example. It doesn't seem to hurt anything, as far as I
> can tell, but I could be wrong about this...

On 08/02/2010 05:20 PM, Willem Jan Palenstijn wrote:
> I haven't looked into what makes liblapack.so appear/disappear yet. Does anyone
> familiar with atlas/lapack know this offhand?

I'm not very familiar with these packages, but if liblapack.so
disappears or does not appear because ATLAS can't find libgfortran, the
steps in

http://trac.sagemath.org/sage_trac/ticket/9356#comment:7

might make it (re)appear. Should we do

$ cd SAGE_LOCAL/lib
$ ln -s libgfortran.so.2 libgfortran.so

or a generalization, in the Fortran package?