sage-5.0.x and OS X 10.7 Lion

[William Stein]

Hi,

A major blocker for Sage-5.0 is supporting OS X (version 10.7 -- the
version that has been out for months now).

Fortunately, it is now "relatively easy" to build sage-5.0.beta1 on OS
X 10.7 with XCode 4.x, and have it start up.

(1) Build [1] as usual.
(2) Apply Simon's patch trac11900_fix_singleton_hash.patch from #11900 [2].
(3) Fix the static Python library problem:
$ cd SAGE_ROOT
$ rm local/lib/python2.7/config/libpython2.7.a
$ cp local/lib/libpython2.7.dylib local/lib/python2.7/config/
$ ./sage -f spkg/standard/pynac-0.2.3.p0.spkg
$ ./sage -ba

I expect there will be numerous subtle test failures though, just as
there were two months ago when John Palmieri built Sage on 10.7 and
ran the test suite. I'll post later about this.

[1] http://sage.math.washington.edu/home/release/sage-5.0.beta1/
[2] http://trac.sagemath.org/sage_trac/ticket/11900
-- William

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[John H Palmieri]

On Monday, January 16, 2012 7:42:49 AM UTC-8, William wrote:

Hi,

A major blocker for Sage-5.0 is supporting OS X (version 10.7 -- the
version that has been out for months now).

Fortunately, it is now "relatively easy" to build sage-5.0.beta1 on OS
X 10.7 with XCode 4.x, and have it start up.

(1) Build [1] as usual.
(2) Apply Simon's patch trac11900_fix_singleton_hash.patch  from #11900 [2].
(3) Fix the static Python library problem:
$ cd SAGE_ROOT
$ rm local/lib/python2.7/config/libpython2.7.a
$ cp local/lib/libpython2.7.dylib local/lib/python2.7/config/
$ ./sage -f spkg/standard/pynac-0.2.3.p0.spkg
$ ./sage -ba

I expect there will be numerous subtle test failures though, just as
there were two months ago when John Palmieri built Sage on 10.7 and
ran the test suite.    I'll post later about this.

I think that pari is still an issue.  I can't get it to pass tests (i.e., when SAGE_CHECK=yes).  I also think that pari is causing at least some of the doctest failures.  There was a possibly relevant thread on the pari mailing list a few months ago:

  <http://pari.math.u-bordeaux.fr/archives/pari-dev-1108/threads.html#00008>

I don't know if the issues have been fixed upstream.

--
John

[William Stein]

On Mon, Jan 16, 2012 at 9:26 AM, John H Palmieri <jhpalm...@gmail.com> wrote:
>
>
> On Monday, January 16, 2012 7:42:49 AM UTC-8, William wrote:
>>
>> Hi,
>>
>> A major blocker for Sage-5.0 is supporting OS X (version 10.7 -- the
>> version that has been out for months now).
>>
>> Fortunately, it is now "relatively easy" to build sage-5.0.beta1 on OS
>> X 10.7 with XCode 4.x, and have it start up.

After building Sage on OS X 10.7, "make test" did this:

writing additional files... genindex search
copying static files... done
dumping search index... done
dumping object inventory... done
build succeeded.
Build finished. The built documents can be found in
/Users/wstein/sage/install/sage-5.0.beta1/devel/sage/doc/output/html/ru/tutorial
local/bin/sage-starts

Testing that Sage starts...
[2012-01-16 09:10:19] Sage version 5.0.beta1, released 2012-01-13
Yes, Sage starts.
. local/bin/sage-env && sage-maketest
/bin/sh: local/bin/sage-env: No such file or directory
make: *** [test] Error 1

---------

Huh? So I guess running "make test" doesn't work right in
sage-5.0.beta1 since sage-env is missing (why?).

William

[Dan Drake]

On Mon, 16 Jan 2012 at 03:43PM -0800, William Stein wrote:
> Testing that Sage starts...
> [2012-01-16 09:10:19] Sage version 5.0.beta1, released 2012-01-13
> Yes, Sage starts.
> . local/bin/sage-env && sage-maketest
> /bin/sh: local/bin/sage-env: No such file or directory
> make: *** [test] Error 1
>
> ---------
>
> Huh? So I guess running "make test" doesn't work right in
> sage-5.0.beta1 since sage-env is missing (why?).

Is it related to http://trac.sagemath.org/sage_trac/ticket/11073 ?

Dan

--
--- Dan Drake
----- http://mathsci.kaist.ac.kr/~drake
-------

[Jeroen Demeyer]

On 2012-01-17 03:24, Dan Drake wrote:
>> Huh? So I guess running "make test" doesn't work right in
>> sage-5.0.beta1 since sage-env is missing (why?).
>
> Is it related to http://trac.sagemath.org/sage_trac/ticket/11073 ?

Yes, it is. Should be fixed now in the latest version of #11073.

[William Stein]

On Mon, Jan 16, 2012 at 3:43 PM, William Stein <wst...@gmail.com> wrote:
> On Mon, Jan 16, 2012 at 9:26 AM, John H Palmieri <jhpalm...@gmail.com> wrote:
>>
>>
>> On Monday, January 16, 2012 7:42:49 AM UTC-8, William wrote:
>>>
>>> Hi,
>>>
>>> A major blocker for Sage-5.0 is supporting OS X (version 10.7 -- the
>>> version that has been out for months now).
>>>
>>> Fortunately, it is now "relatively easy" to build sage-5.0.beta1 on OS
>>> X 10.7 with XCode 4.x, and have it start up.
>
> After building Sage on OS X 10.7, "make test" did this:

For the record, running tests with "sage -t devel/sage/sage" yields
hundred(s) of failing files:

http://wstein.org/home/wstein/tmp/test-sage-5.0.beta1-osx10.7.txt

It could be that most of these boil down to some code at the core of
PARI (the bezout function) being miscompiled.

There is a discussion about this from August 2011 here:

http://pari.math.u-bordeaux.fr/archives/pari-dev-1108/msg00000.html

It unfortunately appears based on the mailing list -- and searching
through the ** 5 months ** since then -- that nothing at all was done
to try to fix the problem, even though I had setup access to 10.7 for
the Pari developers. Probably the main problem is that the machine I
setup -- sqrt5.math.washington.edu -- is on the CS network, and for
some reason it keeps getting kicked off.

This really sucks. Maybe I can rewrite their bezout to get around
the problem.

-- William

[John Cremona]

>> After building Sage on OS X 10.7, "make test" did this:
>
> For the record, running tests with "sage -t devel/sage/sage" yields
> hundred(s) of failing files:
>
>   http://wstein.org/home/wstein/tmp/test-sage-5.0.beta1-osx10.7.txt
>
> It could be that most of these boil down to some code at the core of
> PARI (the bezout function) being miscompiled.
>
> There is a discussion about this from August 2011 here:
>
>   http://pari.math.u-bordeaux.fr/archives/pari-dev-1108/msg00000.html
>
> It unfortunately appears based on the mailing list -- and searching
> through the ** 5 months ** since then -- that nothing at all was done
> to try to fix the problem, even though I had setup access to 10.7 for
> the Pari developers.   Probably the main problem is that the machine I
> setup -- sqrt5.math.washington.edu -- is on the CS network, and for
> some reason it keeps getting kicked off.
>
> This really sucks.   Maybe I can rewrite their bezout to get around
> the problem.

William, I am in the same building as Karim and Bill for the rest of
this week. Can you explain to me what the problem is (*) so I can
talk to them? They have a Pari-Days all next week in Bordeaux and I
may persuade them that this is worth them spending time on.

(*) OK so I read through that thread. Maybe a personal reminder that
this is still outstanding would help...

John

>
>  -- William
>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

[William Stein]

Another option is to turn of optimization (-O0) when building part of
PARI under XCode 4.x, since the problem is bad compiler optimization.
I just tried, export SAGE_DEBUG="yes", then building PARI with
SAGE_CHECK="yes", and I get: "The PARI self-tests all passed".

So something based on optimization flags is the workaround I'll pursue
for now. Something that is slightly slower than optimal is way
better than infinitely slower.

-- William

[William Stein]

On Tue, Jan 17, 2012 at 1:47 PM, John Cremona <john.c...@gmail.com> wrote:
>>> After building Sage on OS X 10.7, "make test" did this:
>>
>> For the record, running tests with "sage -t devel/sage/sage" yields
>> hundred(s) of failing files:
>>
>>   http://wstein.org/home/wstein/tmp/test-sage-5.0.beta1-osx10.7.txt
>>
>> It could be that most of these boil down to some code at the core of
>> PARI (the bezout function) being miscompiled.
>>
>> There is a discussion about this from August 2011 here:
>>
>>   http://pari.math.u-bordeaux.fr/archives/pari-dev-1108/msg00000.html
>>
>> It unfortunately appears based on the mailing list -- and searching
>> through the ** 5 months ** since then -- that nothing at all was done
>> to try to fix the problem, even though I had setup access to 10.7 for
>> the Pari developers.   Probably the main problem is that the machine I
>> setup -- sqrt5.math.washington.edu -- is on the CS network, and for
>> some reason it keeps getting kicked off.
>>
>> This really sucks.   Maybe I can rewrite their bezout to get around
>> the problem.
>
> William, I am in the same building as Karim and Bill for the rest of
> this week.  Can you explain to me what the problem is (*) so I can
> talk to them?  They have a Pari-Days all next week in Bordeaux and I
> may persuade them that this is worth them spending time on.
>
> (*) OK so I read through that thread.  Maybe a personal reminder that
> this is still outstanding would help...

Since this is a compiler bug, I'm not sure what is the best thing to
do at present.

Let me run the full test suite of Sage first, having built PARI with
-O0 and see what happens. If that works, I will just need to figure
out how to build only bezout* with -O0 (or -O1). I'm guessing the
best thing the PARI devs could do at this point is report a bug to
Apple (maybe Justin Walker could file a bug report to Apple about
their compiler)? Also, suggestions about how to properly build only
*one* file with -O1 would be helpful.

-- William

>
> John
>
>
>>
>>  -- William
>>
>> --
>> To post to this group, send an email to sage-...@googlegroups.com
>> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
>> For more options, visit this group at http://groups.google.com/group/sage-devel
>> URL: http://www.sagemath.org
>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

--

[John Cremona]

I would do that by adding a specific target into the Makefile for that
one file with a custom compile command which would then be used
instead of the default pattern to make a .o from a .c. (Several years
ago I had one file which crashed a compiler at some optimazation
level, so I did this to compile just that file at a lower level.)

John

[William Stein]

Thanks John -- good idea.

To follow up on my previous message, when I rebuilt pari with
SAGE_DEBUG="yes" (so -O0), then ran the test suite, only the following
tests fail:

sage -t devel/sage/sage/combinat/schubert_polynomial.py # Killed/crashed
sage -t devel/sage/sage/combinat/tableau.py # Time out
sage -t devel/sage/sage/combinat/root_system/weyl_characters.py # Time out
sage -t devel/sage/sage/combinat/sf/hall_littlewood.py # Time out
sage -t devel/sage/sage/combinat/sf/homogeneous.py # Time out
sage -t devel/sage/sage/combinat/sf/ns_macdonald.py # Time out
sage -t devel/sage/sage/combinat/sf/schur.py # Time out
sage -t devel/sage/sage/combinat/sf/sfa.py # Time out
sage -t devel/sage/sage/matrix/matrix_double_dense.pyx # 4 doctests failed
sage -t devel/sage/sage/matrix/matrix2.pyx # 4 doctests failed
sage -t devel/sage/sage/libs/symmetrica/schur.pxi # Time out
sage -t devel/sage/sage/rings/polynomial/polynomial_element.pyx # 1
doctests failed
sage -t devel/sage/sage/rings/qqbar.py # Time out
sage -t devel/sage/sage/tests/interrupt.pyx # 13 doctests failed

The obvious next thing to do is try building symmetrica with less
optimization and trying the above tests.

-- William

[Justin C. Walker]

On Jan 17, 2012, at 14:04 , William Stein wrote:

> On Tue, Jan 17, 2012 at 1:47 PM, John Cremona <john.c...@gmail.com> wrote:
>>>> After building Sage on OS X 10.7, "make test" did this:

> Let me run the full test suite of Sage first, having built PARI with

> -O0 and see what happens. If that works, I will just need to figure
> out how to build only bezout* with -O0 (or -O1). I'm guessing the
> best thing the PARI devs could do at this point is report a bug to
> Apple (maybe Justin Walker could file a bug report to Apple about
> their compiler)? Also, suggestions about how to properly build only
> *one* file with -O1 would be helpful.

I can certainly do that. It would help to know a little more than the above, since the only "bezout" files I could find are text files that (I don't think) get compiled.

Do we know what ".c" files are involved? Could the problem like with a library (whose initials might once have been GPM)? I have not delved into the innards of pari enough to be able to sort this out quickly on my own.

Anything more specific than "it breaks" will get a lot more attention from Apple, and I think Apple knows that their compiler suite for 10.7 has significant issues.

Justin

--
Justin C. Walker
Curmudgeon-at-large
Director
Institute for the Absorption of Federal Funds
----
186,000 Miles per Second
Not just a good idea:
it's the law!
----

[William Stein]

On Tue, Jan 17, 2012 at 3:20 PM, Justin C. Walker <jus...@mac.com> wrote:
>
> On Jan 17, 2012, at 14:04 , William Stein wrote:
>
>> On Tue, Jan 17, 2012 at 1:47 PM, John Cremona <john.c...@gmail.com> wrote:
>>>>> After building Sage on OS X 10.7, "make test" did this:
>
>> Let me run the full test suite of Sage first, having built PARI with
>> -O0 and see what happens.   If that works, I will just need to figure
>> out how to build only bezout* with -O0 (or -O1).   I'm guessing the
>> best thing the PARI devs could do at this point is report a bug to
>> Apple (maybe Justin Walker could file a bug report to Apple about
>> their compiler)?  Also, suggestions about how to properly build only
>> *one* file with -O1 would be helpful.
>
> I can certainly do that.  It would help to know a little more than the above, since the only "bezout" files I could find are text files that (I don't think) get compiled.
>
> Do we know what ".c" files are involved?  Could the problem like with a library (whose initials might once have been GPM)?  I have not delved into the innards of pari enough to be able to sort this out quickly on my own.
>
> Anything more specific than "it breaks" will get a lot more attention from Apple, and I think Apple knows that their compiler suite for 10.7 has significant issues.

There's a lot of discussion related to your question in the PARI
mailing list thread I linked to. Fortunately, this is something that
can be observed outside of Sage (purely with Pari), and there is even
a variant of it if you don't use GMP (or MPIR). It doesn't look like
GMP/MPIR is the problem by the way.

Incidentally, with both Symmetrica and PARI built using -O0, the only
test failures in devel/sage/sage are:

sage -t devel/sage/sage/matrix/matrix_double_dense.pyx # 4
doctests failed
sage -t devel/sage/sage/matrix/matrix2.pyx # 4 doctests failed
sage -t

devel/sage/sage/rings/polynomial/polynomial_element.pyx # 1 doctests
failed

sage -t devel/sage/sage/tests/interrupt.pyx # 13 doctests failed

The matrix stuff is numerical noise being slightly different.

The polynomial_element.pyx thing looks pretty weird.

The interrupt.pyx looks scary -- somehow the current clever interrupt
handling and counting code isn't working right at all with OS X 10.7.

-- William

[John H Palmieri]

I've been trying to build Sage with OS X Lion over the past few days, and things are going pretty well: there are a few doctest failures, but I think most are numerical noise.  Most spkgs build just fine, but because of bugs in Apple's compiler, a few -- pari, gsl, symmetrica -- have problems unless we modify them.  So I propose the following modifications in the spkg-install scripts for those packages:

 - check to see if running Lion, and if so
 - check to see if /usr/bin/gcc-4.2 exists.  this might be available from an older installation of XCode, or it can be installed using the gcc package Georg posted a link to: http://r.research.att.com/tools/.  If it's there, set CC=gcc-4.2.
 - otherwise, turn off optimization (compile with -O0) for those packages.  This seems to avoid the bugs, although it will slow down those specific pieces of code.

Opinions?

We also seem to need to delete the file SAGE_ROOT/local/lib/python/config/libpython2.7.a.  Any reasons why doing this is a bad idea?  (See <http://trac.sagemath.org/sage_trac/ticket/11967>.)

There are still a few remaining problems with OS X Lion, like self-tests for cvxopt, but maybe someone can figure out how to fix them.

See <http://trac.sagemath.org/sage_trac/ticket/11881> for the trac metaticket about building Sage on Lion.

--
John

[R. Andrew Ohana]

Do we know which files build incorrectly, we should only set the flag
for those (and not the whole library). Also, we should see if -O1
works too.

>
> We also seem to need to delete the file
> SAGE_ROOT/local/lib/python/config/libpython2.7.a.  Any reasons why doing
> this is a bad idea?  (See
> <http://trac.sagemath.org/sage_trac/ticket/11967>.)
>
> There are still a few remaining problems with OS X Lion, like self-tests for
> cvxopt, but maybe someone can figure out how to fix them.
>
> See <http://trac.sagemath.org/sage_trac/ticket/11881> for the trac
> metaticket about building Sage on Lion.
>
> --
> John
>

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to
> sage-devel+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

I've started looking into the difficulties of getting sage to build
with clang (on lion), and have made some progress on the sage
libraries.

Using CC=clang, CXX=clang++, and SAGE_CHECK=yes for all spkgs except

eclib
givaro
libm4rie
gap
lcalc
linbox
matplotlib
polybori
pynac
libfplll
singular
ratpoints
sage

where I used gcc instead (all Xcode 4). Most of the spkgs not listed
here that don't just build only needed some flags set (the exceptions
being python, which needed a patch, and ecm, which will probably need
a patch to the asm).

I've built sage with roughly the same doctest failures as you get when
you turn off optimization for pari and symmetrica, although there are
likely some spkgs that aren't respecting CC and CXX. So far I've found
that flint, flintqs, ratpoints, f2c, and a small part of sage haven't
been, but thanfully all of them have been trivial fixes (except for
the last one, I haven't yet found where exactly it is calling gcc).

(Beyond the issue of fortran) I'm not sure if it will be possible to
build all of the sage libraries with clang. For instance, it currently
doesn't yet support nested functions, which I know at least ratpoints
uses.

Anyway, I just figured I'd share what I've found so far.

--
Andrew

[Volker Braun]

On Tuesday, January 31, 2012 1:01:41 PM UTC-8, R. Andrew Ohana wrote:

(Beyond the issue of fortran) I'm not sure if it will be possible to
build all of the sage libraries with clang. For instance, it currently
doesn't yet support nested functions, which I know at least ratpoints
uses.

C does't have nested functions, this is a GNU extension. Any code that uses them should be rewritten to be standards-compliant C.

 

[R. Andrew Ohana]

On Tue, Jan 31, 2012 at 13:13, Volker Braun <vbrau...@gmail.com> wrote:
> On Tuesday, January 31, 2012 1:01:41 PM UTC-8, R. Andrew Ohana wrote:
>>
>> (Beyond the issue of fortran) I'm not sure if it will be possible to
>> build all of the sage libraries with clang. For instance, it currently
>> doesn't yet support nested functions, which I know at least ratpoints
>> uses.
>
> C does't have nested functions, this is a GNU extension.

True, however clang aims to be gcc compatible, and intends to support
nested functions at some (indeterminate) point in the future.

> Any code that uses
> them should be rewritten to be standards-compliant C.
>
>

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to
> sage-devel+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

--
Andrew

[John H Palmieri]

On Tuesday, January 31, 2012 1:01:41 PM UTC-8, R. Andrew Ohana wrote:

On Mon, Jan 30, 2012 at 10:04, John H Palmieri <jhpalm...@gmail.com> wrote:
>
> On Tuesday, January 17, 2012 1:59:45 PM UTC-8, William wrote:
>>
>> On Tue, Jan 17, 2012 at 1:39 PM, William Stein <wst...@gmail.com> wrote:
>>
>> > On Mon, Jan 16, 2012 at 3:43 PM, William Stein <wst...@gmail.com> wrote:

>> >> On Mon, Jan 16, 2012 at 9:26 AM, John H Palmieri <jhpa...@gmail.com>

I tried that and still got self-test failures.  I didn't try all possible combinations of compiling some files with -O3 and some with -O0 until tests passed.
 

Also, we should see if -O1
works too.

No, it doesn't.  I tried that.

[snip]

I've started looking into the difficulties of getting sage to build
with clang (on lion), and have made some progress on the sage
libraries.

Using CC=clang, CXX=clang++, and SAGE_CHECK=yes for all spkgs except

I thought I had tried this with pari and had failures, but now it seems to work.  In an existing Sage build, installing cvxopt this way doesn't work for me -- I get self test failures -- but maybe setting CC=clang (etc.) and building the whole thing from scratch will work better.

Please keep working on this; it sounds like you're getting somewhere.

--
John

[Volker Braun]

What I'm trying to say is: Upstream needs to be informed that they shouldn't use non-standard C extensions. Nested functions especially are a bad design choice in a world that is moving away from executable stacks.

[Dr. David Kirkby]

On 01/31/12 09:20 PM, R. Andrew Ohana wrote:
> On Tue, Jan 31, 2012 at 13:13, Volker Braun<vbrau...@gmail.com> wrote:
>> On Tuesday, January 31, 2012 1:01:41 PM UTC-8, R. Andrew Ohana wrote:
>>>
>>> (Beyond the issue of fortran) I'm not sure if it will be possible to
>>> build all of the sage libraries with clang. For instance, it currently
>>> doesn't yet support nested functions, which I know at least ratpoints
>>> uses.
>>
>> C does't have nested functions, this is a GNU extension.
>
> True, however clang aims to be gcc compatible, and intends to support
> nested functions at some (indeterminate) point in the future.

But it makes the code unportable. What hope do we have with the Sun/Oracle
compiler if idiots use non-standard C? What hope do we have if we try to build
on Windows at some point in the future using a native compiler? All these GNU
extensions are a headache for everyone except the linux community.

If there was not so much poor code in Sage, building with the Sun compiler would
be possible, but it rejects much of the code.

>> Any code that uses
>> them should be rewritten to be standards-compliant C.

Agreed.

--
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

[R. Andrew Ohana]

On Tue, Jan 31, 2012 at 15:52, Dr. David Kirkby <david....@onetel.net> wrote:
> On 01/31/12 09:20 PM, R. Andrew Ohana wrote:
>>
>> On Tue, Jan 31, 2012 at 13:13, Volker Braun<vbrau...@gmail.com>  wrote:
>>>
>>> On Tuesday, January 31, 2012 1:01:41 PM UTC-8, R. Andrew Ohana wrote:
>>>>
>>>>
>>>> (Beyond the issue of fortran) I'm not sure if it will be possible to
>>>> build all of the sage libraries with clang. For instance, it currently
>>>> doesn't yet support nested functions, which I know at least ratpoints
>>>> uses.
>>>
>>>
>>> C does't have nested functions, this is a GNU extension.
>>
>>
>> True, however clang aims to be gcc compatible, and intends to support
>> nested functions at some (indeterminate) point in the future.
>
>
> But it makes the code unportable. What hope do we have with the Sun/Oracle
> compiler if idiots use non-standard C? What hope do we have if we try to
> build on Windows at some point in the future using a native compiler? All
> these GNU extensions are a headache for everyone except the linux community.
>
> If there was not so much poor code in Sage, building with the Sun compiler
> would be possible, but it rejects much of the code.
>
>
>>> Any code that uses
>>> them should be rewritten to be standards-compliant C.
>
>
> Agreed.

To clarify, I was not arguing for using GNU extensions, I was merely
trying to point out what clang intends to do, and some current
troubles with it. I agree that we should have standards-compliant C,
which is part of the reason I started working on a clang port, since
it is more restrict in this regards that gcc (but less so than other c
compilers).

>
> --
> A: Because it messes up the order in which people normally read text.
> Q: Why is top-posting such a bad thing?
> A: Top-posting.
> Q: What is the most annoying thing in e-mail?
>
>

[R. Andrew Ohana]

Well that is fair :)

>
>>
>> Also, we should see if -O1
>> works too.
>
>
> No, it doesn't.  I tried that.

Too bad

>
> [snip]
>
>> I've started looking into the difficulties of getting sage to build
>> with clang (on lion), and have made some progress on the sage
>> libraries.
>>
>> Using CC=clang, CXX=clang++, and SAGE_CHECK=yes for all spkgs except
>
> I thought I had tried this with pari and had failures, but now it seems to
> work.  In an existing Sage build, installing cvxopt this way doesn't work
> for me -- I get self test failures -- but maybe setting CC=clang (etc.) and
> building the whole thing from scratch will work better.

Hmm, now I'm getting self test failures, however the package seems to
still be installing for me. I wasn't paying attention during the whole
build process, so this may have happened before as well.

>
> Please keep working on this; it sounds like you're getting somewhere.

I plan to, although I think I'm going to move to working on a system
where sage works (such as sage.math), and then hopefully the changes I
make there apply to Lion.

You should continue working on the lion issues. If you need/want,
William has a few lion systems to test/work on, if you would like an
account just let me know.

>
> --
> John
>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to
> sage-devel+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

--
Andrew

[Dima Pasechnik]

On Wednesday, February 1, 2012 5:01:41 AM UTC+8, R. Andrew Ohana wrote:

On Mon, Jan 30, 2012 at 10:04, John H Palmieri <jhpalm...@gmail.com> wrote:
>
> On Tuesday, January 17, 2012 1:59:45 PM UTC-8, William wrote:
>>
>> On Tue, Jan 17, 2012 at 1:39 PM, William Stein <wst...@gmail.com> wrote:
>>
>> > On Mon, Jan 16, 2012 at 3:43 PM, William Stein <wst...@gmail.com> wrote:

>> >> On Mon, Jan 16, 2012 at 9:26 AM, John H Palmieri <jhpa...@gmail.com>

I wonder if this is clang from Xcode 4 (i.e. what their cc is set to), or some other clang?

Dima

[R. Andrew Ohana]

clang --version
Apple clang version 3.0 (tags/Apple/clang-211.12) (based on LLVM 3.0svn)
Target: x86_64-apple-darwin11.2.0
Thread model: posix

cc --version
i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc.
build 5658) (LLVM build 2336.1.00)
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

So this is Xcode4's clang, and cc is pointing to Xcode's llvm-gcc (as
is gcc). Currently I'm working with clang in a gentoo prefix on
sage.math, and am finding a few more issues that don't show up on
lion.

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to
> sage-devel+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

--
Andrew

[Justin C. Walker]

On Feb 1, 2012, at 04:46 , Dima Pasechnik wrote:
>
> On Wednesday, February 1, 2012 5:01:41 AM UTC+8, R. Andrew Ohana wrote:
>>
>> On Mon, Jan 30, 2012 at 10:04, John H Palmieri <jhpalm...@gmail.com> wrote:

[snip]

>> I've started looking into the difficulties of getting sage to build
>> with clang (on lion), and have made some progress on the sage
>> libraries.
>>
>> Using CC=clang, CXX=clang++, and SAGE_CHECK=yes for all spkgs except
>>
>
> I wonder if this is clang from Xcode 4 (i.e. what their cc is set to), or
> some other clang?

I am using Xcode 4.2.1 on 10.7.1. Here's what the compilers think is going on:

$ cc -v
Using built-in specs.
Target: i686-apple-darwin11
Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2336.1~22/src/configure --disable-checking --enable-werror --prefix=/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2336.1~22/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.1.00)

$ clang -v

Apple clang version 3.0 (tags/Apple/clang-211.12) (based on LLVM 3.0svn)

Target: x86_64-apple-darwin11.1.0
Thread model: posix

HTH

Justin

--
Justin C. Walker, Curmudgeon-At-Large
Institute for the Enhancement of the Director's Income
--------
When LuteFisk is outlawed,
Only outlaws will have LuteFisk
--------

[William Stein]

On Mon, Jan 30, 2012 at 10:04 AM, John H Palmieri

Just to clarify, does gcc-4.2 *not* come with the latex XCode 4.x, but
it came with earlier XCode 4.x's? I have gcc-4.2 on my laptop, and
I've never installed anything but XCode 4.x on it.

Just curious.

>  - otherwise, turn off optimization (compile with -O0) for those packages.
> This seems to avoid the bugs, although it will slow down those specific
> pieces of code.

It's probably better to require gcc-4.2 for now if (1) it work, and
(2) is not that hard to install.

-- William

>
> Opinions?
>
> We also seem to need to delete the file
> SAGE_ROOT/local/lib/python/config/libpython2.7.a.  Any reasons why doing
> this is a bad idea?  (See
> <http://trac.sagemath.org/sage_trac/ticket/11967>.)
>
> There are still a few remaining problems with OS X Lion, like self-tests for
> cvxopt, but maybe someone can figure out how to fix them.
>
> See <http://trac.sagemath.org/sage_trac/ticket/11881> for the trac
> metaticket about building Sage on Lion.
>
> --
> John
>

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to
> sage-devel+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

--

[John H Palmieri]

On Friday, February 3, 2012 8:33:53 AM UTC-8, William wrote:

Just to clarify, does gcc-4.2 *not* come with the latex XCode 4.x, but
it came with earlier XCode 4.x's?    I have gcc-4.2 on my laptop, and
I've never installed anything but XCode 4.x on it.

Just curious.

I think that it came with XCode versions before 4.2, but it doesn't come with 4.2.  (I installed 4.1, or maybe 4.0, on a laptop, and have since upgraded to 4.2.  That machine has gcc-4.2.  On another machine, I just installed Lion and then XCode 4.2, and it doesn't have gcc-4.2.)

--
John

[Jonathan Bober]

On Tue, Jan 31, 2012 at 3:52 PM, Dr. David Kirkby <david....@onetel.net> wrote:

But it makes the code unportable. What hope do we have with the Sun/Oracle compiler if idiots use non-standard C? What hope do we have if we try to build on Windows at some point in the future using a native compiler? All these GNU extensions are a headache for everyone except the linux community.

If there was not so much poor code in Sage, building with the Sun compiler would be possible, but it rejects much of the code.

 

This is a long reply, so a quick summary: "If there was not so much poor code in Sage, building with the Sun compiler would be possible, but Sage would be much less useful."

This is a rather late response, but as I don't see any responses that complain

about your viewpoint, it seems appropriate. I don't want anyone to see your

email that think that your views are representative of the majority of

opinions, because I think that they are both wrong and harmful.

Whether or not someone writes standard C has little to do with whether or not

they are an "idiot" and in many (most?) cases it doesn't even have much to do

with whether or not they are ignorant (which is much different). As far as

example given above, I would say that, though I do not know him personally, I

am pretty sure that Michael Stoll (who wrote ratpoints) is not an idiot. (The

situation is probably quite the opposite.)

In general, person X might use nonstandard GNU extension Y for many reasons,

including:

(1) X has more important things to worry about than what is or isn't

standard C/C++.

(2) gcc is (for better or for worse) a de facto standard.

(3) Y happens to be very useful, and perhaps makes the code much faster

(4) X knows that Y is not a standard, but for he/she/they is/are only

interested in running his/her/their code for his/her/their own purposes, and

doesn't much care whether or not it works for Sage, or on Solaris, or on

Windows.

That is (partially) why I think your viewpoint is wrong. Another nice example

is smalljac. It would be very nice if this were included in Sage, bit it won't

be for the moment largely because

"This code is only supported on 64-bit Intel/AMD platforms

(life is too short to spend it worrying about overflowing a 32-bit

integer)."

This has nothing to do with GNU extensions, but you can see quite clearly that

the Drew Sutherland is well aware of some shortcomings of his code and that he

has no intention of fixing them. That puts the onus to fix the code on Sage

developers, if they want it to be included, the same way that Sage developers

should be willing to fix ratpoints if they want it to compile with clang (or

sun's compiler, or MSVC).

For another example: I recently tried to compile some of my own code using

clang++ and discovered that I am not allowed to do

void f(int j) {

    complex<double> x[j];

    [...]

}

even though g++ accepts that. ( See

http://clang.llvm.org/compatibility.html#vla ) Now, I have a few options. I

could allocate everything on the heap explicitly; I could use

vector<complex<double> >; or I could specify a maximum size for j. All of

these options are bad for me, because I care very much about speed, and

I know that j is small, and 99% of the time it will be less that 20, but I

don't want to always allocate space for j = 100 because this is very rare. So

I have to consciously decide what to do, and if I decide to stick with the

nonstandard gcc extension it is not because I am an idiot but because it is

probably the fastest/easiest option on what is likely to be the only

compiler I ever care about.

Continuing: I might find a 5% improvement in my code through proper use of

__builtin_expect on gcc. I know this is not portable, but should I really take

the time to figure out how to write a proper macro that will work for one

million different compilers, when am only going to use one? If I don't it has

nothing to do with idiocy.

As for why your viewpoint might be harmful: I have heard anecdotes of people

not wanting to release their code because it was ugly, or nonstandard, or

difficult to use, etc. As long as the response that they are going to receive

it along the lines of the above, that viewpoint is valid, even if the response

should be "Yes, this is ugly, but it is still awesome."

Large (most?) parts of Sage have simply come from third-party code written

by people who never had any intention of having anything to do with Sage.

Such people should be encouraged to release their code, not called idiots

when they do release it.

(This was a rather long off-topic response, and I'm not going to respond again.

Or at least not in this thread.)

[David Roe]

As for why your viewpoint might be harmful: I have heard anecdotes of people

not wanting to release their code because it was ugly, or nonstandard, or

difficult to use, etc. As long as the response that they are going to receive

it along the lines of the above, that viewpoint is valid, even if the response

should be "Yes, this is ugly, but it is still awesome."

Large (most?) parts of Sage have simply come from third-party code written

by people who never had any intention of having anything to do with Sage.

Such people should be encouraged to release their code, not called idiots

when they do release it.

+1

[Julien Puydt]

Le 04/02/2012 07:39, Jonathan Bober a �crit :

> For another example: I recently tried to compile some of my own code using
> clang++ and discovered that I am not allowed to do
>
> void f(int j) {
> complex<double> x[j];
> [...]
> }
>
> even though g++ accepts that. ( See
> http://clang.llvm.org/compatibility.html#vla ) Now, I have a few options. I
> could allocate everything on the heap explicitly; I could use
> vector<complex<double> >; or I could specify a maximum size for j. All of
> these options are bad for me, because I care very much about speed, and
> I know that j is small, and 99% of the time it will be less that 20, but I
> don't want to always allocate space for j = 100 because this is very
> rare. So
> I have to consciously decide what to do, and if I decide to stick with the
> nonstandard gcc extension it is not because I am an idiot but because it is
> probably the fastest/easiest option on what is likely to be the only
> compiler I ever care about.

If the "j" in question is always a constant in the code (I mean if you
always have "f(2)", "f(156)" and the like), you may perhaps consider a
template :
http://stackoverflow.com/questions/499106/what-does-template-unsigned-int-n-mean

Snark on #sagemath

[Volker Braun]

I agree of course that we should converse in a civil manner.

On Friday, February 3, 2012 10:39:45 PM UTC-8, Jonathan Bober wrote:

In general, person X might use nonstandard GNU extension Y for many reasons,

In my experience, it usually boils down to 

5) Person used language extension without being aware that it is non-standard. 

Often there is a good but highly non-obvious reason for why the cool feature was not accepted in the standard. Case in point: nested C functions. Pointing out the rationale of the standards committee in a polite manner can work wonders.

"This code is only supported on 64-bit Intel/AMD platforms

(life is too short to spend it worrying about overflowing a 32-bit

integer)."

Newsflash: i386 has 64-bit ints as well. With a minimum of effort you can write code that works independent of bitwidth and will reduce your own maintenance and debugging overhead down the road because it makes your assumptions explicit. This isn't the dark ages anymore where all you had was an int and you could only pray that it is long enough.

void f(int j) {

    complex<double> x[j];

    [...]

}

Your snippet is neither valid C99 nor valid C++. Since it seems like you are just using C with std::complex<double>, why not make it proper C99 double complex? Then it would be standards-compliant and perhaps even faster / less likely to trigger compiler bugs. Also, you are hopefully aware of the size limitations of C99 VLAs.

Alternatively, if you want to use C++ then you have various template tricks to make the size statically known. This not only gives you stack allocated arrays but the compiler can also unroll loops at compile time. Also, you are probably underestimating just how well-tuned std::vector is (did you actually time it?). 

Continuing: I might find a 5% improvement in my code through proper use of

__builtin_expect on gcc. I know this is not portable, but should I really take

the time to figure out how to write a proper macro that will work for one

million different compilers, when am only going to use one?

spending the 2 minutes writing a proper macro / inline template will improve the readability of the code and you'll end up ahead in the long run.

[Dima Pasechnik]

Can we get Lion on bsd.math.washington.edu ?

sqrt5 is down again...

Dima

[William Stein]

On Mon, Feb 20, 2012 at 4:28 AM, Dima Pasechnik <dim...@gmail.com> wrote:
> Can we get Lion on bsd.math.washington.edu ?

I could, but then we will no longer have a 10.6 build/test machine, I
think, and that would be bad.
Also, I can't do this until next week, since I'm in San Diego right now.

> sqrt5 is down again...
>
> Dima
>

[Dima Pasechnik]

In gmane.comp.mathematics.sage.devel, you wrote:
> On Mon, Feb 20, 2012 at 4:28 AM, Dima Pasechnik <dim...@gmail.com> wrote:
>> Can we get Lion on bsd.math.washington.edu ?
>
> I could, but then we will no longer have a 10.6 build/test machine, I
> think, and that would be bad.
> Also, I can't do this until next week, since I'm in San Diego right now.
>

sure, I understand, but it seems that 10.7 is becoming a major hassle to
support, and so having a machine running it available would really be
great...

Dima

[R. Andrew Ohana]

I would rather not move bsd.math over to lion. We have a few machines
running lion currently and there is a tendency of crashing -- we don't
want that on bsd. I'm heading in later today to fix one of these
machines from such a crash, and when I do I'll be sure to make you an
account.

FYI, sqrt5 has been down for awhile since it is extremely prone to
kernel panics and I have yet to figure out the issue. My guess at this
point in time is that these are due to it using the 32-bit kernel,
which apple clearly isn't supporting any more -- the main requirement
of 10.8 over 10.7 is support for the 64-bit kernel. Unfortunately this
is a different requirement than simply having 64-bit hardware, since
apple has not and probably will not add 64-bit kernel support for many
of the early core 2 systems, which includes sqrt5's model.

>
> Dima
>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

--
Andrew

[Justin C. Walker]

On Feb 21, 2012, at 08:13 , R. Andrew Ohana wrote:

> On Tue, Feb 21, 2012 at 07:08, Dima Pasechnik <dim...@gmail.com> wrote:
>> In gmane.comp.mathematics.sage.devel, you wrote:
>>> On Mon, Feb 20, 2012 at 4:28 AM, Dima Pasechnik <dim...@gmail.com> wrote:
>>>> Can we get Lion on bsd.math.washington.edu ?
>>>
>>> I could, but then we will no longer have a 10.6 build/test machine, I
>>> think, and that would be bad.
>>> Also, I can't do this until next week, since I'm in San Diego right now.
>>>
>> sure, I understand, but it seems that 10.7 is becoming a major hassle to
>> support, and so having a machine running it available would really be

>> great…

> FYI, sqrt5 has been down for awhile since it is extremely prone to
> kernel panics and I have yet to figure out the issue. My guess at this
> point in time is that these are due to it using the 32-bit kernel,
> which apple clearly isn't supporting any more -- the main requirement
> of 10.8 over 10.7 is support for the 64-bit kernel.

Is this true? My 10.7 system has a 64-bit kernel, at least according to 'file':

$ file /mach_kernel
/mach_kernel: Mach-O universal binary with 2 architectures
/mach_kernel (for architecture x86_64): Mach-O 64-bit executable x86_64
/mach_kernel (for architecture i386): Mach-O executable i386

Justin

--
Justin C. Walker
Director

Institute for the Enhancement of the Director's Income
--

Fame is fleeting, but obscurity
just drags on and on. F&E

[R. Andrew Ohana]

On Tue, Feb 21, 2012 at 10:30, Justin C. Walker <jus...@mac.com> wrote:
>
> On Feb 21, 2012, at 08:13 , R. Andrew Ohana wrote:
>
>> On Tue, Feb 21, 2012 at 07:08, Dima Pasechnik <dim...@gmail.com> wrote:
>>> In gmane.comp.mathematics.sage.devel, you wrote:
>>>> On Mon, Feb 20, 2012 at 4:28 AM, Dima Pasechnik <dim...@gmail.com> wrote:
>>>>> Can we get Lion on bsd.math.washington.edu ?
>>>>
>>>> I could, but then we will no longer have a 10.6 build/test machine, I
>>>> think, and that would be bad.
>>>> Also, I can't do this until next week, since I'm in San Diego right now.
>>>>
>>> sure, I understand, but it seems that 10.7 is becoming a major hassle to
>>> support, and so having a machine running it available would really be
>>> great…
>
>> FYI, sqrt5 has been down for awhile since it is extremely prone to
>> kernel panics and I have yet to figure out the issue. My guess at this
>> point in time is that these are due to it using the 32-bit kernel,
>> which apple clearly isn't supporting any more -- the main requirement
>> of 10.8 over 10.7 is support for the 64-bit kernel.
>
> Is this true?

Positive:

$ uname -pr
11.3.0 i386

> My 10.7 system has a 64-bit kernel, at least according to 'file':
>
> $ file /mach_kernel
> /mach_kernel: Mach-O universal binary with 2 architectures
> /mach_kernel (for architecture x86_64): Mach-O 64-bit executable x86_64
> /mach_kernel (for architecture i386):   Mach-O executable i386

OSX has included extensions in the 32-bit kernel to execute 64-bit
code since leopard (or potentially earlier), this only indicates that
the kernel + hardware supports executing 64-bit code.

>
> Justin
>
> --
> Justin C. Walker
> Director
> Institute for the Enhancement of the Director's Income
> --
> Fame is fleeting, but obscurity
>   just drags on and on.      F&E
>
>
>

[Justin C. Walker]

On Feb 21, 2012, at 14:16 , R. Andrew Ohana wrote:

> On Tue, Feb 21, 2012 at 10:30, Justin C. Walker <jus...@mac.com> wrote:
>>
>> On Feb 21, 2012, at 08:13 , R. Andrew Ohana wrote:

[snip]

>>> FYI, sqrt5 has been down for awhile since it is extremely prone to
>>> kernel panics and I have yet to figure out the issue. My guess at this
>>> point in time is that these are due to it using the 32-bit kernel,
>>> which apple clearly isn't supporting any more -- the main requirement
>>> of 10.8 over 10.7 is support for the 64-bit kernel.
>>
>> Is this true?
>
> Positive:
>
> $ uname -pr
> 11.3.0 i386
>
>> My 10.7 system has a 64-bit kernel, at least according to 'file':
>>
>> $ file /mach_kernel
>> /mach_kernel: Mach-O universal binary with 2 architectures
>> /mach_kernel (for architecture x86_64): Mach-O 64-bit executable x86_64
>> /mach_kernel (for architecture i386): Mach-O executable i386
>
> OSX has included extensions in the 32-bit kernel to execute 64-bit
> code since leopard (or potentially earlier), this only indicates that
> the kernel + hardware supports executing 64-bit code.

I don't think this is really correct. This is what the Apple website says:

=========
These Macs use the 64-bit kernel by default in Mac OS X v10.6.

• Mac Pro (Mid 2010)
• MacBook Pro (Early 2011)
• iMac (21.5-inch and 27-inch, Mid 2011)
=========

I'm running the MacBook Pro (Early 2011). 'uname -pr' says what you report, but I think that is the "basic ISP" architecture, which is, and I assume will be, i386.

The kernel being "x86_64" means that it actually runs in 64-bit mode.

Justin

--
Justin C. Walker, Curmudgeon-At-Large

Institute for the Absorption of Federal Funds
--------
If you're not confused,
You're not paying attention
--------

[R. Andrew Ohana]

Yes, although for lion it seems to be a much larger number of systems.

>
> I'm running the MacBook Pro (Early 2011).  'uname -pr'

Looking at another system that has booted with the 64-bit kernel it
seems that -p does not distinguish, but -m does:

sqrt5:~ uname -m
i386

sage1:~ uname -m
x86_64

'file /mach_kernel' matches on these two systems

> says what you report, but I think that is the "basic ISP" architecture, which is, and I assume will be, i386.
>
> The kernel being  "x86_64" means that it actually runs in 64-bit mode.
>
> Justin
>
> --
> Justin C. Walker, Curmudgeon-At-Large
> Institute for the Absorption of Federal Funds
> --------
> If you're not confused,
> You're not paying attention
> --------
>
>
>

[Justin C. Walker]

Not sure about that, but I don't know why that would be.

>>
>> I'm running the MacBook Pro (Early 2011). 'uname -pr'

To be clear: I am running 10.7 on this MacBook.

> Looking at another system that has booted with the 64-bit kernel it
> seems that -p does not distinguish, but -m does:
>
> sqrt5:~ uname -m
> i386
>
> sage1:~ uname -m
> x86_64

Yeah; the "-p" is the ISP architecture (the machine "api"): i386 is the common ISP architecture for (most) Intel processors; "-m" is the hardware type (in my case: Core i7), x86_64.

> 'file /mach_kernel' matches on these two systems

That's because Apple ships a single (universal) kernel binary that is intended to work on all supported systems.

Cheers,

Justin

--
Justin C. Walker, Curmudgeon-at-Large
() The ASCII Ribbon Campaign
/\ Help Cure HTML Email

[Robert Bradshaw]

+1

I was having the exact same thoughts when reading this thread, and
this is not the first time. Sometimes it feels like I'm in a math
lecture and someone is complaining that it's no good because of the
poor penmanship. "Sheesh, he can't even properly cross his T's, and
will you look at that punctuation..." Perhaps that's a bit extreem,
but you get the point. It's not that there's not room for improvement
(there often is plenty), but that's not the primary consideration, if
it's a consideration at all for the original author. Even for those
that intend to write good code (which is most of the people I know),
experience, ignorance, or other constraints get in the way.

For someone who's profession is coding, a higher standard is more
reasonable, but I think we'd be waiting around a long time before
anyone wrote ratpoints if we wait for the "professionals." More
mathematicians know (a little) how to code than programmers know (a
little) algebraic geometry, and they have a lot to contribute.

- Robert