Adding GCC and MPC as standard packages

100 views
Skip to first unread message

Jeroen Demeyer

unread,
Mar 11, 2012, 6:15:35 PM3/11/12
to sage-devel
I have made a spkg for GCC (GNU compiler collection) version 4.6.3 with
compilers for C, C++ and Fortran, see Trac #12369.

The GCC spkg depends on an MPC (multi-precision complex numbers) spkg.
There has been an optional MPC package and a Sage interface for it. GCC
would use the upgraded MPC package from #12515.

According to (unwritten?) rules, one cannot add a spkg as standard
package if it hasn't been an optional spkg first. I would like to ask
for an exception for GCC and add it immediately as standard package in
sage-5.0. Also MPC would need to become a standard package, but this
already has been an optional package and so is less controversial.

Let me clarify that GCC would not always be built: if the system already
contains sufficiently recent versions of gcc, g++ and gfortran (or g95
or g77), then GCC would not be built. Currently, "sufficiently recent
version of gcc" is defined as: at least 4.4.0 but not 4.6.0 nor 4.6.1
(there are known bugs in cliquer and rubiks with these gcc versions).
An environment variable SAGE_INSTALL_GCC can be used to override this
default choice (set to "yes" to force installation of GCC or to "no" to
prevent it).

The Fortran package (which contained gfortran binaries for OS X) would
be removed, therefore about 4MB would be added to the source tarball
size (Fortran was 33MB, GCC+MPC is 37MB).

I think there is not much point in making GCC optional, since it's only
used at build-time. Once Sage has been built, you don't need it anymore.

What are your +1/-1 to making GCC and MPC standard packages?


Jeroen.

Florent Hivert

unread,
Mar 11, 2012, 6:34:36 PM3/11/12
to sage-...@googlegroups.com
Hi,

Isn't it used for compiling Cython code ? IF so it could be called from inside
the notebook using "%cython".

> What are your +1/-1 to making GCC and MPC standard packages?

+1

Florent

William Stein

unread,
Mar 11, 2012, 11:19:59 PM3/11/12
to sage-...@googlegroups.com
+1.  (and I *hate* adding packages)


>
> Florent
>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

Julien Puydt

unread,
Mar 12, 2012, 2:58:06 AM3/12/12
to sage-...@googlegroups.com
Le dimanche 11 mars, William Stein a écrit:

> +1. (and I *hate* adding packages)

As it's optional, it's not really added: +1 too.

And I hate packages...

Snark on #sagemath

Jeroen Demeyer

unread,
Mar 12, 2012, 4:29:08 AM3/12/12
to sage-...@googlegroups.com
On 2012-03-11 23:34, Florent Hivert wrote:
> Isn't it used for compiling Cython code ?
Of course it is indeed. But compiling Cython code is not the main use
I had in mind for this package.

Keshav Kini

unread,
Mar 12, 2012, 10:14:06 AM3/12/12
to sage-...@googlegroups.com
Julien Puydt <julien...@laposte.net> writes:
> As it's optional, it's not really added: +1 too.

But it is not optional - Jeroen is proposing to make this a standard
package. Or did I misunderstand you?

-Keshav

----
Join us in #sagemath on irc.freenode.net !

Julien Puydt

unread,
Mar 12, 2012, 10:48:08 AM3/12/12
to sage-...@googlegroups.com
Le lundi 12 mars, Keshav Kini a écrit:

> Julien Puydt <julien...@laposte.net> writes:
> > As it's optional, it's not really added: +1 too.
>
> But it is not optional - Jeroen is proposing to make this a standard
> package. Or did I misunderstand you?

Poor choice of words on my part : it is not "optional" since it's
"standard", but it's "optional" since it's "not mandatory". It is that
second meaning I had in mind.

It will be like the fortran package, which checks for the presence of
a convenient fortran compiler and sets things up to use it if found.

which makes me think it might be interesting if packages had an uniform
way to pass such information around : fortran has SAGE_FORTRAN and
makes hackish symlinks ; the atlas package also has magical
environment variables and symlinks tricks...

Snark on #sagemath

Volker Braun

unread,
Mar 12, 2012, 2:48:51 PM3/12/12
to sage-...@googlegroups.com
I'm all in favor of adding the gcc spkg. 

One thing that I'm not entirely happy with is that we bulid gcc without graphite/ppl/cloog, so I suspect that the code speed isn't as good as it could be. But since we only use the self-compiled gcc if the os-supplied one is crap I think its fine.

John H Palmieri

unread,
Mar 12, 2012, 3:00:06 PM3/12/12
to sage-...@googlegroups.com


On Sunday, March 11, 2012 3:15:35 PM UTC-7, Jeroen Demeyer wrote:

What are your +1/-1 to making GCC and MPC standard packages?

+1 (especially since it allows us to build Sage on Lion without making other compromises)

--
John

Jeroen Demeyer

unread,
Mar 12, 2012, 4:44:21 PM3/12/12
to sage-...@googlegroups.com
Well, I believe Graphite is used only when you supply specific command
line options to gcc (-O3 is not sufficient). Since those options are
rarely/never supplied by the various SPKGs, I don't think there is much
to gain by adding CLooG.

François Bissey

unread,
Mar 12, 2012, 4:58:38 PM3/12/12
to sage-...@googlegroups.com
Correct. The options you get with graphite are:
-floop-interchange
-floop-strip-mine
-floop-block
there is also
-fgraphite-identity
but I believe it is available when you have both graphite and lto at the same
time.

I tested the graphite optimization on some lattice QCD code that has many
levels of looping and the benefits were quite significant (around 30%). I was
extremely impressed.

Francois

Simon King

unread,
Mar 13, 2012, 2:33:57 AM3/13/12
to sage-devel
Hi François, Jeroen and Volker,

On 12 Mrz., 21:58, François Bissey <francois.bis...@canterbury.ac.nz>
wrote:
> On Mon, 12 Mar 2012 21:44:21 Jeroen Demeyer wrote:> On 2012-03-12 19:48, Volker Braun wrote:
> > > One thing that I'm not entirely happy with is that we bulid gcc without
> > > graphite/ppl/cloog, ...
>
> > Well, I believe Graphite is used only when you supply specific command
> > line options to gcc (-O3 is not sufficient).  Since those options are
> > rarely/never supplied by the various SPKGs, I don't think there is much
> > to gain by adding CLooG.
>
> ...
> I tested the graphite optimization on some lattice QCD code that has many
> levels of looping and the benefits were quite significant (around 30%). I was
> extremely impressed.

That sounds like
- We should add graphite/ppl/cloog to the new gcc spkg
- The spkg should be used not only if the system's gcc has the
"wrong" version, but also if the system's gcc does not provide
graphite/ppl/cloog
- We should see what spkgs would benefit from the additional
optimization, and let their spkg-install / their makefiles provide the
relevant gcc command line options
- We should see what Cython modules may benefit from the additional
optimization, and add gcc command line options in module_list.py as
appropriate.

Isn't it?

Cheers,
Simon

William Stein

unread,
Mar 13, 2012, 2:40:42 AM3/13/12
to sage-...@googlegroups.com
On Mon, Mar 12, 2012 at 11:33 PM, Simon King <simon...@uni-jena.de> wrote:
> Hi François, Jeroen and Volker,
>
> On 12 Mrz., 21:58, François Bissey <francois.bis...@canterbury.ac.nz>
> wrote:
>> On Mon, 12 Mar 2012 21:44:21 Jeroen Demeyer wrote:> On 2012-03-12 19:48, Volker Braun wrote:
>> > > One thing that I'm not entirely happy with is that we bulid gcc without
>> > > graphite/ppl/cloog, ...
>>
>> > Well, I believe Graphite is used only when you supply specific command
>> > line options to gcc (-O3 is not sufficient).  Since those options are
>> > rarely/never supplied by the various SPKGs, I don't think there is much
>> > to gain by adding CLooG.
>>
>> ...
>> I tested the graphite optimization on some lattice QCD code that has many
>> levels of looping and the benefits were quite significant (around 30%). I was
>> extremely impressed.
>
> That sounds like
>  - We should add graphite/ppl/cloog to the new gcc spkg

We could also wait to do this until later, but keep it in mind...

>  - The spkg should be used not only if the system's gcc has the
> "wrong" version, but also if the system's gcc does not provide
> graphite/ppl/cloog
>  - We should see what spkgs would benefit from the additional
> optimization, and let their spkg-install / their makefiles provide the
> relevant gcc command line options
>  - We should see what Cython modules may benefit from the additional
> optimization, and add gcc command line options in module_list.py as
> appropriate.
>
> Isn't it?
>
> Cheers,
> Simon
>

Julien Puydt

unread,
Mar 13, 2012, 3:08:36 AM3/13/12
to sage-...@googlegroups.com
Le lundi 12 mars, Simon King a écrit:

> - The spkg should be used not only if the system's gcc has the
> "wrong" version, but also if the system's gcc does not provide
> graphite/ppl/cloog

Is sage a distribution of mathematical software or a distribution?

It's already replicating a good chunk of my ARM setup (it mostly lacks
the kernel, emacs and latex)!

I voted +1 for gcc as a standard non-mandatory (wink to Keshav)
package ; I vote -1 for yet another standard mandatory package.

Snark on #sagemath

Jeroen Demeyer

unread,
Mar 13, 2012, 3:41:35 AM3/13/12
to sage-...@googlegroups.com
On 2012-03-13 07:33, Simon King wrote:
> - We should see what spkgs would benefit from the additional
> optimization, and let their spkg-install / their makefiles provide the
> relevant gcc command line options

And who is volunteering to do this? Not me. This sounds like something
which can be done by Volker's compilerwrapper.

If you do this, you certainly should not restrict yourself to the
Graphite command line options, you should optimize gcc's flags in
general (like adding -march for example).

Jeroen.

P Purkayastha

unread,
Mar 13, 2012, 4:18:41 AM3/13/12
to sage-...@googlegroups.com


On Tuesday, March 13, 2012 3:41:35 PM UTC+8, Jeroen Demeyer wrote:

If you do this, you certainly should not restrict yourself to the
Graphite command line options, you should optimize gcc's flags in
general (like adding -march for example).

Jeroen.

 
If you do set -march, then -march=native is a pretty safe option. gcc picks up many optimization flags by itself after detecting the cpu. I have used this option for more than 3 years now for my whole Gentoo linux setup.

But (of course) binaries made from -march=native can not be distributed.

Simon King

unread,
Mar 13, 2012, 4:40:06 AM3/13/12
to sage-devel
Hi!

On 13 Mrz., 07:33, Simon King <simon.k...@uni-jena.de> wrote:
> That sounds like
>  - We should add graphite/ppl/cloog to the new gcc spkg

Google told me that graphite is in gcc since version 4.4. But from the
comments in this thread I got the impression that graphite is
something that one may or may not have on top of gcc.

So, could some expert please elaborate why it is claimed that the gcc
spkg does *not* come with graphite? I suppose gcc 4.6.3 (provided by
the spkg) contains graphite, since gcc 4.4 does.

Cheers,
Simon

Jeroen Demeyer

unread,
Mar 13, 2012, 4:45:43 AM3/13/12
to sage-...@googlegroups.com
On 2012-03-13 09:40, Simon King wrote:
> So, could some expert please elaborate why it is claimed that the gcc
> spkg does *not* come with graphite? I suppose gcc 4.6.3 (provided by
> the spkg) contains graphite, since gcc 4.4 does.
Graphite depends on PPL (which is in Sage) and on CLooG (which is NOT in
Sage). So, while the Graphite loop optimizer is in GCC, it will not be
compiled in without these dependencies.

Simon King

unread,
Mar 13, 2012, 4:50:55 AM3/13/12
to sage-devel
Hi Jeroen,

On 13 Mrz., 09:45, Jeroen Demeyer <jdeme...@cage.ugent.be> wrote:
> Graphite depends on PPL (which is in Sage) and on CLooG (which is NOT in
> Sage).  So, while the Graphite loop optimizer is in GCC, it will not be
> compiled in without these dependencies.

Thank you!

So, can I assume that when I install PPL and CLooG on my machine, then
my system's gcc would be able to use the loop optimizer? Or would I
then also need to rebuild gcc?
Or: If the user happens to have CLooG, would our gcc spkg
automatically support the loop optimization?

And if I'd like to test something like "-march=native" mentioned by
Purkayastha, can I set some environment variable such that building a
Cython extension would be built with -march=native?

Cheers,
Simon

Simon King

unread,
Mar 13, 2012, 4:59:41 AM3/13/12
to sage-devel
Hi Jeroen,

On 13 Mrz., 09:50, Simon King <simon.k...@uni-jena.de> wrote:
> And if I'd like to test something like "-march=native" mentioned by
> Purkayastha, can I set some environment variable such that building a
> Cython extension would be built with -march=native?

Got it: With CFLAGS="-march=native", it seems to work.

I have just installed CLooG, and you said that Sage provides PPL. So,
can I expect that it would make sense to add -march=native and -floop-
interchange
-floop-strip-mine -floop-block to CFLAGS, and run sage -ba, in order
to see whether Sage would become noticeably faster?

Cheers,
Simon

Jeroen Demeyer

unread,
Mar 13, 2012, 5:00:52 AM3/13/12
to sage-...@googlegroups.com
On 2012-03-13 09:50, Simon King wrote:
> So, can I assume that when I install PPL and CLooG on my machine, then
> my system's gcc would be able to use the loop optimizer? Or would I
> then also need to rebuild gcc?
Yes, you need to rebuild GCC.

> Or: If the user happens to have CLooG, would our gcc spkg
> automatically support the loop optimization?

No. I guess we could support an optional CLooG spkg in Sage. But I
first want to concentrate on having a basic GCC spkg which works (it
still needs_review at #12369). Extra features like this can still be
added later.

> And if I'd like to test something like "-march=native" mentioned by
> Purkayastha, can I set some environment variable such that building a
> Cython extension would be built with -march=native?

Try CFLAGS="-O3 -march=native" CXXFLAGS="-O3 -march=native" and see if
that works.

Jeroen Demeyer

unread,
Mar 13, 2012, 5:07:19 AM3/13/12
to sage-...@googlegroups.com
On 2012-03-13 09:59, Simon King wrote:
> I have just installed CLooG, and you said that Sage provides PPL.
Where did you install CLooG? If you install it in $SAGE_ROOT/local,
then it might "just work" if you compile the GCC spkg from #12369.
Check the "configure" part of spkg/logs/gcc-4.6.3 for CLooG.

Simon King

unread,
Mar 13, 2012, 5:27:43 AM3/13/12
to sage-devel
Hi Jeroen,

On 13 Mrz., 10:07, Jeroen Demeyer <jdeme...@cage.ugent.be> wrote:
> On 2012-03-13 09:59, Simon King wrote:> I have just installed CLooG, and you said that Sage provides PPL.
>
> Where did you install CLooG?

By "install", I meant "install with Yast" (my laptop runs under
openSuse).

Cheers,
Simon

Jeroen Demeyer

unread,
Mar 13, 2012, 5:36:57 AM3/13/12
to sage-...@googlegroups.com
On 2012-03-13 10:27, Simon King wrote:
> By "install", I meant "install with Yast" (my laptop runs under
> openSuse).
Does the GCC spkg find it?

Simon King

unread,
Mar 13, 2012, 5:39:36 AM3/13/12
to sage-devel
No idea yet.

I the last minutes, I tried to follow your advice to install CLooG in
$SAGE_LOCAL. However, it requires gmp. So, I tried "sage -i gmp".
However, installation of the gmp package failed.

Cheers,
Simon

Simon King

unread,
Mar 13, 2012, 6:01:20 AM3/13/12
to sage-devel
Hi!

I created a ticket for the broken (at least on openSuse) optional
gmp-4.2.1 package: #12661.

Cheers,
Simon

Jeroen Demeyer

unread,
Mar 13, 2012, 6:43:20 AM3/13/12
to sage-...@googlegroups.com
You should be able to use MPIR instead of GMP.

But maybe there is no need to install CLooG in $SAGE_LOCAL.

Simon King

unread,
Mar 13, 2012, 7:39:52 AM3/13/12
to sage-devel
Hi Jeroen,

On 13 Mrz., 11:43, Jeroen Demeyer <jdeme...@cage.ugent.be> wrote:
> You should be able to use MPIR instead of GMP.

I was told by CLooG that it needs GMP.

Independent of that, I wouldn't mind to fix the optional gmp package.
I got it to install work on my openSUSE. Unfortunately, spkg-check
fails, and also I have to recall how to add a patch to an spkg.

Actually I am now trying to upgrade the optional package to gmp-5.0.4
- so far, it seems to build, and it already contains a patch that the
old gmp-4.2.1 spkg would need in order to build on openSuse.

> But maybe there is no need to install CLooG in $SAGE_LOCAL.

We will see later...

Cheers,
Simon

François Bissey

unread,
Mar 13, 2012, 4:54:32 PM3/13/12
to sage-...@googlegroups.com
On Tue, 13 Mar 2012 01:18:41 P Purkayastha wrote:
> On Tuesday, March 13, 2012 3:41:35 PM UTC+8, Jeroen Demeyer wrote:
> > If you do this, you certainly should not restrict yourself to the
> > Graphite command line options, you should optimize gcc's flags in
> > general (like adding -march for example).
> >
> > Jeroen.
>
> If you do set -march, then *-march=native* is a pretty safe option. gcc

> picks up many optimization flags by itself after detecting the cpu. I have
> used this option for more than 3 years now for my whole Gentoo linux setup.
>
> But (of course) binaries made from -march=native can not be distributed.

-march=native only works on x86* other archs like ppc(64) are not supported.

Francois

Dima Pasechnik

unread,
Mar 13, 2012, 8:11:00 PM3/13/12
to sage-...@googlegroups.com


On Monday, 12 March 2012 06:15:35 UTC+8, Jeroen Demeyer wrote:
I have made a spkg for GCC (GNU compiler collection) version 4.6.3 with
compilers for C, C++ and Fortran, see Trac #12369.

The GCC spkg depends on an MPC (multi-precision complex numbers) spkg.
There has been an optional MPC package and a Sage interface for it.  GCC
would use the upgraded MPC package from #12515.

According to (unwritten?) rules, one cannot add a spkg as standard
package if it hasn't been an optional spkg first.  I would like to ask
for an exception for GCC and add it immediately as standard package in
sage-5.0.  Also MPC would need to become a standard package, but this
already has been an optional package and so is less controversial.

Let me clarify that GCC would not always be built: if the system already
contains sufficiently recent versions of gcc, g++ and gfortran (or g95
or g77), then GCC would not be built.  Currently, "sufficiently recent
version of gcc" is defined as: at least 4.4.0 but not 4.6.0 nor 4.6.1
(there are known bugs in cliquer and rubiks with these gcc versions).

$ gcc -v
Using built-in specs.
Target: i686-apple-darwin10
[...]
gcc version 4.2.1 (Apple Inc. build 5666) (dot 3)

You seem to indicate this will not be used to build Sage any more.
Right?
(even though this is a working setup, as far as Sage 5.0.beta* are concerned...

Thanks,
Dima

 

An environment variable SAGE_INSTALL_GCC can be used to override this
default choice (set to "yes" to force installation of GCC or to "no" to
prevent it).

The Fortran package (which contained gfortran binaries for OS X) would
be removed, therefore about 4MB would be added to the source tarball
size (Fortran was 33MB, GCC+MPC is 37MB).

I think there is not much point in making GCC optional, since it's only
used at build-time.  Once Sage has been built, you don't need it anymore.

What are your +1/-1 to making GCC and MPC standard packages?


Jeroen.

Dima Pasechnik

unread,
Mar 13, 2012, 8:16:24 PM3/13/12
to sage-...@googlegroups.com
your post also seems to say that the effort to port Sage to xcode4 is stopped.
As also on darwin 11 the default compiler does not satisfy the version condition
that you mention.

Please clarify.

John H Palmieri

unread,
Mar 13, 2012, 9:07:17 PM3/13/12
to sage-...@googlegroups.com

In my experience, it's not a working setup on OS X Lion: pari fails self-tests (and this also leads to failures in Sage doctests), gsl fails self-tests, and symmetrica also causes failures in Sage doctests.  See also #12459.  (See #11881 for the meta ticket for building on Lion.)

I guess the answer to your question is yes, though: even with OS X 10.6 or earlier, the proposal is to build gcc.

--
John

leif

unread,
Mar 14, 2012, 12:47:00 AM3/14/12
to sage-devel
On Mar 12, 9:58 pm, François Bissey <francois.bis...@canterbury.ac.nz>
wrote:
> On Mon, 12 Mar 2012 21:44:21 Jeroen Demeyer wrote:> On 2012-03-12 19:48, Volker Braun wrote:
> > > I'm all in favor of adding the gcc spkg.
>
> > > One thing that I'm not entirely happy with is that we bulid gcc without
> > > graphite/ppl/cloog, so I suspect that the code speed isn't as good as it
> > > could be. But since we only use the self-compiled gcc if the os-supplied
> > > one is crap I think its fine.
>
> > Well, I believe Graphite is used only when you supply specific command
> > line options to gcc (-O3 is not sufficient).  Since those options are
> > rarely/never supplied by the various SPKGs, I don't think there is much
> > to gain by adding CLooG.

Well, you can set CFLAGS; meanwhile most spkgs should respect user
settings as well.

PPL is also part of Sage (although I don't recall right now whether
it's a suitable version; 0,11 and 0.12 should be fine for GCC/CLooG),
and CLooG[-PPL] itself is quite small (756KB for 0.15.11, which is the
version currently needed by GCC).


-leif


P.S.: In general +1, although IMHO there should also be a Sage tarball
without all these potentially unused spkgs.

P.P,S.: I suggested offering an optional GCC spkg (and provided an
experimental 4.5.1 one IIRC) last year; the feedback was less
positive... :-)

Dima Pasechnik

unread,
Mar 14, 2012, 1:14:20 AM3/14/12
to sage-...@googlegroups.com
how much binary incompatibility is it going to introduce?
I imagine it's easy to reach the point when system libraries won't work any more
together with binaries produced by such a compiler...

Dima 

--
John

leif

unread,
Mar 14, 2012, 1:25:23 AM3/14/12
to sage-devel
Well, for GCC you'll need a specific version (currently / for GCC
4.6.3 and PPL 0.12: CLooG 0.15.11), and if I'm not completely wrong,
you have to explicitly specify '--with-ppl' (optionally giving the
path prefix where it is installed, if not [directly] below '/usr')
when configuring it, since CLooG supports different backends, while
GCC requires PPL.

GCC (4.6.3) /should/ use (i.e., get built with support for) PPL+CLooG
automatically if its 'configure' happens to find suitable versions of
both.


-leif

IMPORTANT NOTE: If you set CFLAGS when [re]building the Sage library,
you always HAVE TO add '-fno-strict-aliasing', since Python's CFLAGS
(which contain it) get completely dropped in that case, and with
strict aliasing rules GCC produces wrong code for (at least a couple
of) Cython modules.

P Purkayastha

unread,
Mar 14, 2012, 1:44:03 AM3/14/12
to sage-...@googlegroups.com
Interesting. I didn't know this. It seems -march=native is not present on arm and -march is itself not available on ppc.

Simon King

unread,
Mar 14, 2012, 2:24:20 AM3/14/12
to sage-devel
Hi Leif, hi all,

On 14 Mrz., 05:47, leif <not.rea...@online.de> wrote:
> ...
> PPL is also part of Sage

But then, GCC can not use CLooG/PPL (I have tried). I was told that it
is because of a missing header.

The header is easy to get, though: The current (standard) ppl spkg
only builds the c++ interface, but not the c interface. In order to
build it, one just needs to add two letters to ppl's spkg-install.

Here is my experience, so far:

* CLooG needs GMP. The optional GMP spkg is broken (at least on
openSuse), even though it is massively patched, to address problems on
various platforms. I suggest to upgrade the optional GMP spkg to the
latest upstream release: For me, it works WITHOUT patches and passes
all self test. See #12661.

* After installing GMP, installation of CLooG works. I created an
spkg, based on unpatched upstream sources. See #12666.

* The PPL spkg needs to be slightly changed, so that it provides
ppl_c.h (currently, it only provides ppl.hh). I've tried it locally,
there is no ticket yet.

* Then, the new GCC spkg installs fine -- but it can *still* not use
CLooG/PPL!! The install log of the package says:
checking for installed CLooG PPL Legacy... no

In other words, building the PPL with C interface is not enough.

> P.P,S.: I suggested offering an optional GCC spkg (and provided an
> experimental 4.5.1 one IIRC) last year; the feedback was less
> positive... :-)

I guess one important difference is that we have now the trouble with
OS X, that might be solvable with a GCC spkg.

Cheers,
Simon

Francois Bissey

unread,
Mar 14, 2012, 2:27:36 AM3/14/12
to sage-...@googlegroups.com

Yep, good old -mcpu like in the good old days:

frb15@p2n14-c /hpc/home/frb15 :gcc -v

Using built-in specs.

Target: powerpc64-suse-linux

Configured with: ../configure --prefix=/usr --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,java --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.3 --enable-ssp --disable-libssp --with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch --enable-version-specific-runtime-libs --program-suffix=-4.3 --enable-linux-futex --without-system-libunwind --with-cpu=power4 --enable-secureplt --with-long-double-128 --build=powerpc64-suse-linux

Thread model: posix

gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux)

 



leif

unread,
Mar 14, 2012, 2:34:00 AM3/14/12
to sage-devel
IIRC, '-march=native' (on Intel / AMD x86[_64] processors, which have
a CPUID insn; likewise '-mtune=native') requires GCC >= 4,2.1; on PPC,
you [only] have '-mcpu=...' and '-mtune=...' (while at least '-
march=foo' implies '-mtune=foo').

Of course "native" only works as expected (or gives better
performance) if the GCC present "knows" your CPU, i.e. has timing
parameters for it (and probably knows newly added instructions or ISA
extensions).

Since OS distros and their packages are supposed to run on a broad
variety of processors, their GCCs are configured (and all binary
packages supplied are built) with quite generic (and hence suboptimal)
settings. Also, tools like valgrind do not always know the latest
processors (including all instructions generated by a more recent
GCC).


-leif

Jeroen Demeyer

unread,
Mar 14, 2012, 4:21:26 AM3/14/12
to sage-...@googlegroups.com
On 2012-03-14 01:11, Dima Pasechnik wrote:
> You seem to indicate this will not be used to build Sage any more.
> Right?
Exactly.

> (even though this is a working setup, as far as Sage 5.0.beta* are
> concerned...

Well, is there anything wrong with replacing a working setup by a
different working setup? It is likely that GCC-4.6.3 produces better
optimized machine code than GCC-4.2.1, so Sage will probably run
slightly faster.

Jeroen Demeyer

unread,
Mar 14, 2012, 4:25:16 AM3/14/12
to sage-...@googlegroups.com
On 2012-03-14 02:07, John H Palmieri wrote:
> I guess the answer to your question is yes, though: even with OS X 10.6
> or earlier, the proposal is to build gcc.

Since Xcode doesn't include a Fortran compiler and the Fortran spkg will
be removed, we *have* to build GCC anyway for gfortran.

Jeroen Demeyer

unread,
Mar 14, 2012, 4:27:21 AM3/14/12
to sage-...@googlegroups.com
On 2012-03-14 07:24, Simon King wrote:
> * CLooG needs GMP.
Are you sure it doesn't work with MPIR?

Florent Hivert

unread,
Mar 14, 2012, 4:35:59 AM3/14/12
to sage-...@googlegroups.com

Incidentally, C�dric Bastoul, the author of CLooG is working is the same
research lab as me and teaching is the same department as Nicolas. So if close
contact is needed we could help.

Cheers,

Florent

Simon King

unread,
Mar 14, 2012, 4:46:58 AM3/14/12
to sage-devel
Hi Jeroen,

On 14 Mrz., 09:27, Jeroen Demeyer <jdeme...@cage.ugent.be> wrote:
> On 2012-03-14 07:24, Simon King wrote:>  * CLooG needs GMP.
>
> Are you sure it doesn't work with MPIR?

When I first tried to install CLooG, then it failed and told me that
GMP is missing. That's why I said GLooG needs GMP.

However, I could imagine that it would have worked with some
configuration option for CLooG, such as --with-mpir.

Anyway. It could be that cloog is actually not enough; there also is a
cloog-ppl package. What I can say is:
If the ppl spkg is built with C interface and the update of the gmp
spkg is installed and cloog-ppl is built with the configuration option
--with-ppl (yes, cloog-ppl still needs the option --with-ppl, believe
it or not), then your gcc package can use CLooG PPL.

Perhaps the gcc package would also be happy if one just had cloog with
the option --with-ppl (cloog is different from cloog-ppl). Since I now
have both cloog and cloog-ppl installed in my version of
sage-5.0.beta7, I can't be sure.

Unfortunately it seems that sage.math is still down, so that I can't
post the update of the standard ppl spkg (enabling C interface, not
just C++), the update of my cloog spkg (see #12666) and my new cloog-
ppl spkg (will be posted at #12666 as well).

Cheers,
Simon

Jeroen Demeyer

unread,
Mar 14, 2012, 4:52:45 AM3/14/12
to sage-...@googlegroups.com
On 2012-03-14 09:46, Simon King wrote:
> cloog is different from cloog-ppl
I understand it that CLooG is the logical continuation of CLooG-PPL, so
it is the same software. CLooG-PPL 0.15 was followed by CLooG 0.16.

The reason they dropped the "-PPL" suffix is that CLooG 0.16 doesn't
need PPL as build-time dependency. CLooG-PPL needs PPL at build-time
(that's why you need the --with-ppl option). GCC with Graphite needs
PPL itself and CLooG(-PPL) 0.15 or later.

Simon King

unread,
Mar 14, 2012, 5:17:51 AM3/14/12
to sage-devel
Hi Jeroen,

On 14 Mrz., 09:52, Jeroen Demeyer <jdeme...@cage.ugent.be> wrote:
> The reason they dropped the "-PPL" suffix is that CLooG 0.16 doesn't
> need PPL as build-time dependency.  CLooG-PPL needs PPL at build-time
> (that's why you need the --with-ppl option).  GCC with Graphite needs
> PPL itself and CLooG(-PPL) 0.15 or later.

The first thing I tried was CLooG 0.17.0, not using --with-ppl. That
has not been enough for your gcc spkg.

Cheers,
Simon

Simon King

unread,
Mar 14, 2012, 5:25:19 AM3/14/12
to sage-devel
On 14 Mrz., 10:17, Simon King <simon.k...@uni-jena.de> wrote:
> The first thing I tried was CLooG 0.17.0, not using --with-ppl. That
> has not been enough for your gcc spkg.

To be precise:
PPL with C interface and CLooG 0.17.0 not using --with-ppl => no
success
Adding CLooG-PPL 0.15.11 not using --with-ppl => no success
Adding CLooG-PPL 0.15.11 using --with-ppl => success
Adding CLooG 0.17.0 using --with-ppl => still success, but I am not
sure if this is an artefact of the previous step.

Cheers,
Simon

leif

unread,
Mar 14, 2012, 11:13:17 PM3/14/12
to sage-devel
From the GCC prerequisites page:


CLooG-PPL version( 0.15 or CLooG 0.16
Necessary to build GCC with the Graphite loop optimizations. There
are two versions available. CLooG-PPL 0.15 as well as CLooG 0.16. The
former is the default right now. It can be downloaded from
ftp://gcc.gnu.org/pub/gcc/infrastructure/ as cloog-ppl-0.15.tar.gz.

CLooG 0.16 support is still in testing stage, but will be the
default in future GCC releases. It is also available at
ftp://gcc.gnu.org/pub/gcc/infrastructure/ as cloog-0.16.1.tar.gz. To
use it add the additional configure option --enable-cloog-backend=isl.
Even if CLooG 0.16 does not use PPL, PPL is still required for
Graphite.

In both cases --with-cloog configure option should be used if
CLooG is not installed in your default library search path.



Note that there are a couple of 0.15.x and 0.16.x versions floating
around; e.g. CLooG-PPL 0.15.9 doesn't work with PPL 0.11 or 0.12 just
because its 'configure' script expects *exactly* PPL version 0.10; as
mentioned, the latest 0.15.x version of CLooG-PPL (0.15.11) works with
both PPL 0.11 and 0.12, provided you specify (at least) '--with-ppl'.
(Otherwise 'configure' searches for and finds PPL, but doesn't use it
for whatever reason..., but we'd use '--with-ppl=$SAGE_ROOT/local'
anyway.)

So what we currently need is CLooG-PPL 0.15.11, and a suitable version
of PPL. (I'm not 100% sure that PPL 0.11.2, the version currently
shipped with Sage, works, but /I think/(tm) it will...)


-leif

Simon King

unread,
Mar 15, 2012, 2:12:18 AM3/15/12
to sage-devel
Hi Leif, hi all,

On 15 Mrz., 04:13, leif <not.rea...@online.de> wrote:
> So what we currently need is CLooG-PPL 0.15.11, and a suitable version
> of PPL.  (I'm not 100% sure that PPL 0.11.2, the version currently
> shipped with Sage, works, but /I think/(tm) it will...)

I installed
* a modified PPL 0.11.2 spkg, that also installs a C interface,
* a CLooG-PPL 0.15.11 spkg
* a modified gcc 4.6.3 spkg, that builds gcc with graphite. TODO:
Attempt to build graphite only if CLooG-PPL is present.

I defined
CFLAGS="-O3 -march=native -floop-interchange -floop-strip-mine -floop-
block -fno-strict-aliasing"
export CFLAGS
CXXFLAGS="-O3 -march=native -floop-interchange -floop-strip-mine -
floop-block -fno-strict-aliasing"
export CXXFLAGS

And then I rebuilt all of Sage (not only sage -ba, I rebuilt all
spkgs). I did not run the test suite yet, but the compilation in this
high optimization went smoothly on my openSuse laptop, and Sage
starts.

I am looking forward to the total time required for running sage -
testall compared with and without optimization!

Cheers,
Simon

Simon King

unread,
Mar 15, 2012, 2:15:22 AM3/15/12
to sage-devel
On 15 Mrz., 07:12, Simon King <simon.k...@uni-jena.de> wrote:
> I installed
> * a modified PPL 0.11.2 spkg, that also installs a C interface,
> * a CLooG-PPL 0.15.11 spkg
> * a modified gcc 4.6.3 spkg, that builds gcc with graphite.

And I forgot: I also upgraded the optional GMP spkg (GMP is required
by CLooG). Our old spkg was broken and needed massive patches. I made
an spkg based on unpatched GMP 5.0.4 upstream sources. I have no idea
what would happen on other platforms, but it worked on openSuse.

Cheers,
Simon

leif

unread,
Mar 15, 2012, 3:17:33 AM3/15/12
to sage-devel
I'm almost sure CLooG works with MPIR (2.x, with '--enable-gmpcompat',
which we pass) as well.


-leif

Simon King

unread,
Mar 15, 2012, 4:15:09 AM3/15/12
to sage-devel
Hi Leif,

On 15 Mrz., 08:17, leif <not.rea...@online.de> wrote:
> > And I forgot: I also upgraded the optional GMP spkg (GMP is required
> > by CLooG). Our old spkg was broken and needed massive patches. I made
> > an spkg based on unpatched GMP 5.0.4 upstream sources. I have no idea
> > what would happen on other platforms, but it worked on openSuse.
>
> I'm almost sure CLooG works with MPIR (2.x, with '--enable-gmpcompat',
> which we pass) as well.

How? I mean, what options does one need to pass when one wants to
build CLooG-PPL with MPIR? It does not pick it up automatically - when
I first tried to build CLooG (not CLooG-PPL) then it failed because it
missed GMP.

Cheers,
Simon

Dima Pasechnik

unread,
Mar 15, 2012, 5:42:00 AM3/15/12
to sage-...@googlegroups.com


On Thursday, 15 March 2012 16:15:09 UTC+8, Simon King wrote:
Hi Leif,

On 15 Mrz., 08:17, leif <not.rea...@online.de> wrote:
> > And I forgot: I also upgraded the optional GMP spkg (GMP is required
> > by CLooG). Our old spkg was broken and needed massive patches. I made
> > an spkg based on unpatched GMP 5.0.4 upstream sources. I have no idea
> > what would happen on other platforms, but it worked on openSuse.
>
> I'm almost sure CLooG works with MPIR (2.x, with '--enable-gmpcompat',
> which we pass) as well. 

IMHO at least MPIR 1.* does not implement all the GMP functionality, i.e. it is not really
a drop-in replacement. E.g. GAP 4.5 does not work with MPIR, even though it
does work with GMP 5.0.4.
Does MPIR 2.* really implement more GMP calls than MPIR 1.* ?

Simon King

unread,
Mar 15, 2012, 9:10:03 AM3/15/12
to sage-devel
Hi All!

On 13 Mrz., 10:00, Jeroen Demeyer <jdeme...@cage.ugent.be> wrote:
> Try CFLAGS="-O3 -march=native" CXXFLAGS="-O3 -march=native" and see if
> that works.

Meanwhile I managed to build the (slightly modified) gcc with
graphite, based on
* an extension of the standard ppl spkg ("build the ppl C interface",
#12672, needs review)
* an upgrade of the optional gmp spkg (the current version is broken
for me, but 5.0.4 works out of the box. See #12661, needs review.
* a cloog-ppl spkg (note that cloog itself won't suffice). See
#12666, needs review.

Then, building all of Sage (all spkgs) with
CFLAGS="-O3 -march=native -floop-interchange -floop-strip-mine -
floop-block -fno-strict-aliasing"
CXXFLAGS="-O3 -march=native -floop-interchange -floop-strip-mine -
floop-block -fno-strict-aliasing"
went smoothly (of course, some packages override the flags).

And it even seems that the doc tests pass.

And now the bad news: Using the above compiler flags, Sage became
MASSIVELY slower, by factors of 2 or 3. That's why I interrupted sage -
testall after about 54% (there was no error up to that point).

Do you have any insight why optimization has such a big negative
impact?

Even though the optimization thingy seems to be a failure, I still
think that we should upgrade the optional gmp spkg (it is broken, at
least on my machine), and we should have an optional CLooG-PPL package
(which would then require to make our standard PPL spkg build the C
interface, rather than just the C++ interface).

Cheers,
Simon

Simon King

unread,
Mar 16, 2012, 2:20:59 AM3/16/12
to sage-devel
Hi all,

On 15 Mrz., 14:10, Simon King <simon.k...@uni-jena.de> wrote:
> And now the bad news: Using the above compiler flags, Sage became
> MASSIVELY slower, by factors of 2 or 3. That's why I interrupted sage -
> testall after about 54% (there was no error up to that point).

It was argued that this was due to replacing MPIR by GMP.

I am now trying again, after I learnt how to build CLooG-PPL with
MPIR.

However, there is one critical point: Building GCC (with Graphite)
takes longer than 6 hours on my laptop! That is actually longer than
building the rest of Sage.

Therefore, I am still +1 for having it an optional package, but -10
for using it if all what we need is just a Fortran compiler ( I guess
building a fortran compiler works faster than 6 hours), and -100 for
building it by default on all platforms.

Cheers,
Simon

leif

unread,
Mar 16, 2012, 3:51:38 AM3/16/12
to sage-devel
On Mar 16, 7:20 am, Simon King <simon.k...@uni-jena.de> wrote:
> Hi all,
>
> On 15 Mrz., 14:10, Simon King <simon.k...@uni-jena.de> wrote:
>
> > And now the bad news: Using the above compiler flags, Sage became
> > MASSIVELY slower, by factors of 2 or 3. That's why I interrupted sage -
> > testall after about 54% (there was no error up to that point).
>
> It was argued that this was due to replacing MPIR by GMP.
>
> I am now trying again, after I learnt how to build CLooG-PPL with
> MPIR.
>
> However, there is one critical point: Building GCC (with Graphite)
> takes longer than 6 hours on my laptop! That is actually longer than
> building the rest of Sage.

Really? Compiling GCC 4.6.3 ('make -j4') with C, C++, Fortran, Java
and LTO enabled took exactly 3 hours on my dual-core Netbook(!) (AMD
Fusion E-450), plus <4 minutes 'make install'. (Bootstrapped with GCC
4.4.3, '-march=native -O3 -g'.)

Of course this is IMHO still much time for "regular" builds, but as I
understand it, only MacOS users (and others with really "outdated"/
buggy GCC versions) will suffer... XD

To me it's ok to *include* GCC by default (replacing the Fortran
package, such that the tarball doesn't grow much, although we could
offer a tarball without it as well), but to only build it if really
necessary or requested by the user.

I'm pretty sure we can find ways to avoid rebuilding GCC each time one
builds a new Sage version, i.e., create scripts (or introduce
environment variables) to easily recycle an already built GCC.


-leif

Simon King

unread,
Mar 16, 2012, 4:45:35 AM3/16/12
to sage-devel
Hi Leif,

On 16 Mrz., 08:51, leif <not.rea...@online.de> wrote:
> Really? Compiling GCC 4.6.3 ('make -j4') with C, C++, Fortran, Java
> and LTO enabled took exactly 3 hours on my dual-core Netbook(!) (AMD
> Fusion E-450), plus <4 minutes 'make install'.  (Bootstrapped with GCC
> 4.4.3, '-march=native -O3 -g'.)

I have just 'make', without '-j', Intel(R) Core(TM)2 Duo CPU
P7450 @ 2.13GHz, bootstrapped with gcc 4.6.2, '-march=native -O3'.

It finished just a minute ago!
real 509m19.210s
user 420m16.548s
sys 31m48.017s
Successfully installed gcc-4.6.3

When I tried to build it the very first time (bootstrapped with gcc
4.6.2, WITHOUT graphite), it was
real 117m26.849s
user 105m11.818s
sys 9m39.734s

When I used the resulting gcc 4.6.3 (without graphite) to bootstrap
another gcc 4.6.3 *with* graphite based on GMP (not MPIR), it was
something like
real 152m18.098s
user 138m19.662s
sys 9m16.678s

Now, I am using cloog-ppl built with MPIR. I wonder if that is the
reason why it takes longer?
Anyway. I will post the install logs to the ticket.

With the gcc that I first built, I could make Sage with '-march=native
-floop* -O3', but the result was inacceptably slow code. But it was
argued that it might be due to GMP. Now, I hope that I will get
amazingly fast code with MPIR...!

> To me it's ok to *include* GCC by default (replacing the Fortran
> package, such that the tarball doesn't grow much, although we could
> offer a tarball without it as well), but to only build it if really
> necessary or requested by the user.

The question is: Is a missing fortran compiler a good reason to build
all of GCC?

> I'm pretty sure we can find ways to avoid rebuilding GCC each time one
> builds a new Sage version, i.e., create scripts (or introduce
> environment variables) to easily recycle an already built GCC.

Sure!

Best regards,
Simon

Jeroen Demeyer

unread,
Mar 16, 2012, 4:51:20 AM3/16/12
to sage-...@googlegroups.com
On 2012-03-16 09:45, Simon King wrote:
> The question is: Is a missing fortran compiler a good reason to build
> all of GCC?
In fact, yes. Because building "just" a Fortran compiler implies that
most of GCC must be built anyway.
Reply all
Reply to author
Forward
0 new messages